Percentile-based residuals for model assessment

Research output: Contribution to journalArticlepeer-review

Abstract

Residuals are a key component of diagnosing model fit. The usual practice is to compute standardized residuals using expected values and standard deviations of the observed data, then use these values to detect outliers and assess model fit. Approximate normality of these residuals is key for this process to have good properties, but in many modeling contexts, especially for complex, multi-level models, normality may not hold. In these cases outlier detection and model diagnostics aren't properly calibrated. Alternatively, as we demonstrate, residuals computed from the percentile location of a datum's value in its full predictive distribution lead to well calibrated evaluations of model fit. We generalize an approach described by Dunn and Smyth (1996) and evaluate properties mathematically, via case-studies and by simulation. In addition, we show that the standard residuals can be calibrated to mimic the percentile approach, but that this extra step is avoided by directly using percentile-based residuals. For both the percentile-based residuals and the calibrated standard residuals, the use of full predictive distributions with the appropriate location, spread and shape is necessary for valid assessments.

Original languageEnglish (US)
JournalUnknown Journal
StatePublished - Oct 8 2019

Keywords

  • Model assessment
  • Non-Gaussian predictions
  • Outlier detection
  • Percentile-based residuals
  • Well-calibrated diagnostics

ASJC Scopus subject areas

  • General

Fingerprint Dive into the research topics of 'Percentile-based residuals for model assessment'. Together they form a unique fingerprint.

Cite this