Diagnostic Measures for Model Criticism

Cinzia Carota, Giovanni Parmigiani, Nicholas G. Polson

Research output: Contribution to journalArticle

Abstract

We discuss the problem of model criticism, with emphasis on developing summary diagnostic measures. We approach model criticism by identifying possible troublesome features of the currently entertained model, embedding the model in an elaborated model, and measuring the value of elaborating. This requires three elements: a model elaboration, a prior distribution, and a utility function. Each triplet generates a different diagnostic measure. We focus primarily on the measure given by a Kullback-Leibler divergence between the marginal prior and posterior distributions on the elaboration parameter. We also develop a linearized version of this diagnostic and use it to show that our procedure is related to other tools commonly used for model diagnostics, such as Bayes factors and the score function. One attraction of this approach is that it allows model criticism to be performed jointly with parameter inference and prediction. Also, this diagnostic approach aims at maintaining an exploratory nature to the criticism process, while affording feasibility of implementation. In this article we present the general outlook and discuss general families of elaborations for use in practice; the exponential connection elaboration plays a key role. We then describe model elaborations for use in diagnosing: departures from normality, goodness of fit in generalized linear models, and variable selection in regression and outlier detection. We illustrate our approach with two applications.

Original languageEnglish (US)
Pages (from-to)753-762
Number of pages10
JournalJournal of the American Statistical Association
Volume91
Issue number434
StatePublished - Jun 1996
Externally publishedYes

Keywords

  • Bayes factor
  • Extra-Poisson variability
  • Hierarchical models
  • Information
  • Model influence
  • Outliers
  • Prior influence
  • Score function

ASJC Scopus subject areas

  • Mathematics(all)
  • Statistics and Probability

Fingerprint Dive into the research topics of 'Diagnostic Measures for Model Criticism'. Together they form a unique fingerprint.

  • Cite this

    Carota, C., Parmigiani, G., & Polson, N. G. (1996). Diagnostic Measures for Model Criticism. Journal of the American Statistical Association, 91(434), 753-762.