Monte Carlo assessments of goodness-of-fit for ecological simulation models

Lance A. Waller, David Smith, James E. Childs, Leslie A. Real

Research output: Contribution to journalArticle

Abstract

One often develops stochastic ecologic simulation models based on local interactions between individuals or groups and bases systemic conclusions on trends summarized over multiple data sets generated from the model. In many cases, such models generate data sets ("realizations") each violating the usual assumptions associated with traditional statistical tests of goodness-of-fit, most notably that of independent observations. Monte Carlo hypothesis tests applied to multiple realizations from such models provide appropriate goodness-of-fit tests regardless of within-model peculiarities. The Monte Carlo tests address the question "Do the observed data appear consistent with the model?" in contrast to the usual question "Does the model appear consistent with the observed data?". In addition, such tests can make use of the same data sets used to draw systemic inference (i.e. the tests require no additional simulation runs). We illustrate the concept using Pearson's chi-square statistic with correlated data. We also consider the behavior of a similar statistic and of "modeling efficiency" in assessing the fit of a simulation model for the spatial spread of raccoon rabies in Connecticut.

Original languageEnglish (US)
Pages (from-to)49-63
Number of pages15
JournalEcological Modelling
Volume164
Issue number1
DOIs
StatePublished - Jun 1 2003
Externally publishedYes

Fingerprint

simulation models
simulation
testing
statistics
rabies
Procyon lotor
statistical analysis
test
modeling

Keywords

  • Goodness-of-fit
  • Model assessment
  • Model validation
  • Modeling efficiency
  • Simulation

ASJC Scopus subject areas

  • Ecology, Evolution, Behavior and Systematics
  • Ecological Modeling
  • Ecology

Cite this

Monte Carlo assessments of goodness-of-fit for ecological simulation models. / Waller, Lance A.; Smith, David; Childs, James E.; Real, Leslie A.

In: Ecological Modelling, Vol. 164, No. 1, 01.06.2003, p. 49-63.

Research output: Contribution to journalArticle

Waller, Lance A. ; Smith, David ; Childs, James E. ; Real, Leslie A. / Monte Carlo assessments of goodness-of-fit for ecological simulation models. In: Ecological Modelling. 2003 ; Vol. 164, No. 1. pp. 49-63.
@article{09b5c6485ea4402a9a61ca6d431d655c,
title = "Monte Carlo assessments of goodness-of-fit for ecological simulation models",
abstract = "One often develops stochastic ecologic simulation models based on local interactions between individuals or groups and bases systemic conclusions on trends summarized over multiple data sets generated from the model. In many cases, such models generate data sets ({"}realizations{"}) each violating the usual assumptions associated with traditional statistical tests of goodness-of-fit, most notably that of independent observations. Monte Carlo hypothesis tests applied to multiple realizations from such models provide appropriate goodness-of-fit tests regardless of within-model peculiarities. The Monte Carlo tests address the question {"}Do the observed data appear consistent with the model?{"} in contrast to the usual question {"}Does the model appear consistent with the observed data?{"}. In addition, such tests can make use of the same data sets used to draw systemic inference (i.e. the tests require no additional simulation runs). We illustrate the concept using Pearson's chi-square statistic with correlated data. We also consider the behavior of a similar statistic and of {"}modeling efficiency{"} in assessing the fit of a simulation model for the spatial spread of raccoon rabies in Connecticut.",
keywords = "Goodness-of-fit, Model assessment, Model validation, Modeling efficiency, Simulation",
author = "Waller, {Lance A.} and David Smith and Childs, {James E.} and Real, {Leslie A.}",
year = "2003",
month = "6",
day = "1",
doi = "10.1016/S0304-3800(03)00011-5",
language = "English (US)",
volume = "164",
pages = "49--63",
journal = "Ecological Modelling",
issn = "0304-3800",
publisher = "Elsevier",
number = "1",

}

TY - JOUR

T1 - Monte Carlo assessments of goodness-of-fit for ecological simulation models

AU - Waller, Lance A.

AU - Smith, David

AU - Childs, James E.

AU - Real, Leslie A.

PY - 2003/6/1

Y1 - 2003/6/1

N2 - One often develops stochastic ecologic simulation models based on local interactions between individuals or groups and bases systemic conclusions on trends summarized over multiple data sets generated from the model. In many cases, such models generate data sets ("realizations") each violating the usual assumptions associated with traditional statistical tests of goodness-of-fit, most notably that of independent observations. Monte Carlo hypothesis tests applied to multiple realizations from such models provide appropriate goodness-of-fit tests regardless of within-model peculiarities. The Monte Carlo tests address the question "Do the observed data appear consistent with the model?" in contrast to the usual question "Does the model appear consistent with the observed data?". In addition, such tests can make use of the same data sets used to draw systemic inference (i.e. the tests require no additional simulation runs). We illustrate the concept using Pearson's chi-square statistic with correlated data. We also consider the behavior of a similar statistic and of "modeling efficiency" in assessing the fit of a simulation model for the spatial spread of raccoon rabies in Connecticut.

AB - One often develops stochastic ecologic simulation models based on local interactions between individuals or groups and bases systemic conclusions on trends summarized over multiple data sets generated from the model. In many cases, such models generate data sets ("realizations") each violating the usual assumptions associated with traditional statistical tests of goodness-of-fit, most notably that of independent observations. Monte Carlo hypothesis tests applied to multiple realizations from such models provide appropriate goodness-of-fit tests regardless of within-model peculiarities. The Monte Carlo tests address the question "Do the observed data appear consistent with the model?" in contrast to the usual question "Does the model appear consistent with the observed data?". In addition, such tests can make use of the same data sets used to draw systemic inference (i.e. the tests require no additional simulation runs). We illustrate the concept using Pearson's chi-square statistic with correlated data. We also consider the behavior of a similar statistic and of "modeling efficiency" in assessing the fit of a simulation model for the spatial spread of raccoon rabies in Connecticut.

KW - Goodness-of-fit

KW - Model assessment

KW - Model validation

KW - Modeling efficiency

KW - Simulation

UR - http://www.scopus.com/inward/record.url?scp=0037883678&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0037883678&partnerID=8YFLogxK

U2 - 10.1016/S0304-3800(03)00011-5

DO - 10.1016/S0304-3800(03)00011-5

M3 - Article

AN - SCOPUS:0037883678

VL - 164

SP - 49

EP - 63

JO - Ecological Modelling

JF - Ecological Modelling

SN - 0304-3800

IS - 1

ER -