Forecasting Emergency Department Crowding: An External, Multicenter Evaluation

Nathan R. Hoot, Stephen K. Epstein, Todd L. Allen, Spencer S. Jones, Kevin M. Baumlin, Neal Chawla, Anna T. Lee, Jesse M. Pines, Amandeep K. Klair, Bradley D. Gordon, Thomas J. Flottemesch, Larry J. LeBlanc, Ian Jones, Scott Levin, Chuan Zhou, Cynthia S. Gadd, Dominik Aronsky

Research output: Contribution to journalArticle

Abstract

Study objective: We apply a previously described tool to forecast emergency department (ED) crowding at multiple institutions and assess its generalizability for predicting the near-future waiting count, occupancy level, and boarding count. Methods: The ForecastED tool was validated with historical data from 5 institutions external to the development site. A sliding-window design separated the data for parameter estimation and forecast validation. Observations were sampled at consecutive 10-minute intervals during 12 months (n=52,560) at 4 sites and 10 months (n=44,064) at the fifth. Three outcome measures-the waiting count, occupancy level, and boarding count-were forecast 2, 4, 6, and 8 hours beyond each observation, and forecasts were compared with observed data at corresponding times. The reliability and calibration were measured following previously described methods. After linear calibration, the forecasting accuracy was measured with the median absolute error. Results: The tool was successfully used for 5 different sites. Its forecasts were more reliable, better calibrated, and more accurate at 2 hours than at 8 hours. The reliability and calibration of the tool were similar between the original development site and external sites; the boarding count was an exception, which was less reliable at 4 of 5 sites. Some variability in accuracy existed among institutions; when forecasting 4 hours into the future, the median absolute error of the waiting count ranged between 0.6 and 3.1 patients, the median absolute error of the occupancy level ranged between 9.0% and 14.5% of beds, and the median absolute error of the boarding count ranged between 0.9 and 2.8 patients. Conclusion: The ForecastED tool generated potentially useful forecasts of input and throughput measures of ED crowding at 5 external sites, without modifying the underlying assumptions. Noting the limitation that this was not a real-time validation, ongoing research will focus on integrating the tool with ED information systems.

Original languageEnglish (US)
JournalAnnals of Emergency Medicine
Volume54
Issue number4
DOIs
StatePublished - Oct 2009

Fingerprint

Calibration
Hospital Emergency Service
Information Systems
Observation
Outcome Assessment (Health Care)
Research

ASJC Scopus subject areas

  • Emergency Medicine

Cite this

Hoot, N. R., Epstein, S. K., Allen, T. L., Jones, S. S., Baumlin, K. M., Chawla, N., ... Aronsky, D. (2009). Forecasting Emergency Department Crowding: An External, Multicenter Evaluation. Annals of Emergency Medicine, 54(4). https://doi.org/10.1016/j.annemergmed.2009.06.006

Forecasting Emergency Department Crowding : An External, Multicenter Evaluation. / Hoot, Nathan R.; Epstein, Stephen K.; Allen, Todd L.; Jones, Spencer S.; Baumlin, Kevin M.; Chawla, Neal; Lee, Anna T.; Pines, Jesse M.; Klair, Amandeep K.; Gordon, Bradley D.; Flottemesch, Thomas J.; LeBlanc, Larry J.; Jones, Ian; Levin, Scott; Zhou, Chuan; Gadd, Cynthia S.; Aronsky, Dominik.

In: Annals of Emergency Medicine, Vol. 54, No. 4, 10.2009.

Research output: Contribution to journalArticle

Hoot, NR, Epstein, SK, Allen, TL, Jones, SS, Baumlin, KM, Chawla, N, Lee, AT, Pines, JM, Klair, AK, Gordon, BD, Flottemesch, TJ, LeBlanc, LJ, Jones, I, Levin, S, Zhou, C, Gadd, CS & Aronsky, D 2009, 'Forecasting Emergency Department Crowding: An External, Multicenter Evaluation', Annals of Emergency Medicine, vol. 54, no. 4. https://doi.org/10.1016/j.annemergmed.2009.06.006
Hoot, Nathan R. ; Epstein, Stephen K. ; Allen, Todd L. ; Jones, Spencer S. ; Baumlin, Kevin M. ; Chawla, Neal ; Lee, Anna T. ; Pines, Jesse M. ; Klair, Amandeep K. ; Gordon, Bradley D. ; Flottemesch, Thomas J. ; LeBlanc, Larry J. ; Jones, Ian ; Levin, Scott ; Zhou, Chuan ; Gadd, Cynthia S. ; Aronsky, Dominik. / Forecasting Emergency Department Crowding : An External, Multicenter Evaluation. In: Annals of Emergency Medicine. 2009 ; Vol. 54, No. 4.
@article{6f7393e718124de48c01e5d7c7c22a34,
title = "Forecasting Emergency Department Crowding: An External, Multicenter Evaluation",
abstract = "Study objective: We apply a previously described tool to forecast emergency department (ED) crowding at multiple institutions and assess its generalizability for predicting the near-future waiting count, occupancy level, and boarding count. Methods: The ForecastED tool was validated with historical data from 5 institutions external to the development site. A sliding-window design separated the data for parameter estimation and forecast validation. Observations were sampled at consecutive 10-minute intervals during 12 months (n=52,560) at 4 sites and 10 months (n=44,064) at the fifth. Three outcome measures-the waiting count, occupancy level, and boarding count-were forecast 2, 4, 6, and 8 hours beyond each observation, and forecasts were compared with observed data at corresponding times. The reliability and calibration were measured following previously described methods. After linear calibration, the forecasting accuracy was measured with the median absolute error. Results: The tool was successfully used for 5 different sites. Its forecasts were more reliable, better calibrated, and more accurate at 2 hours than at 8 hours. The reliability and calibration of the tool were similar between the original development site and external sites; the boarding count was an exception, which was less reliable at 4 of 5 sites. Some variability in accuracy existed among institutions; when forecasting 4 hours into the future, the median absolute error of the waiting count ranged between 0.6 and 3.1 patients, the median absolute error of the occupancy level ranged between 9.0{\%} and 14.5{\%} of beds, and the median absolute error of the boarding count ranged between 0.9 and 2.8 patients. Conclusion: The ForecastED tool generated potentially useful forecasts of input and throughput measures of ED crowding at 5 external sites, without modifying the underlying assumptions. Noting the limitation that this was not a real-time validation, ongoing research will focus on integrating the tool with ED information systems.",
author = "Hoot, {Nathan R.} and Epstein, {Stephen K.} and Allen, {Todd L.} and Jones, {Spencer S.} and Baumlin, {Kevin M.} and Neal Chawla and Lee, {Anna T.} and Pines, {Jesse M.} and Klair, {Amandeep K.} and Gordon, {Bradley D.} and Flottemesch, {Thomas J.} and LeBlanc, {Larry J.} and Ian Jones and Scott Levin and Chuan Zhou and Gadd, {Cynthia S.} and Dominik Aronsky",
year = "2009",
month = "10",
doi = "10.1016/j.annemergmed.2009.06.006",
language = "English (US)",
volume = "54",
journal = "Annals of Emergency Medicine",
issn = "0196-0644",
publisher = "Mosby Inc.",
number = "4",

}

TY - JOUR

T1 - Forecasting Emergency Department Crowding

T2 - An External, Multicenter Evaluation

AU - Hoot, Nathan R.

AU - Epstein, Stephen K.

AU - Allen, Todd L.

AU - Jones, Spencer S.

AU - Baumlin, Kevin M.

AU - Chawla, Neal

AU - Lee, Anna T.

AU - Pines, Jesse M.

AU - Klair, Amandeep K.

AU - Gordon, Bradley D.

AU - Flottemesch, Thomas J.

AU - LeBlanc, Larry J.

AU - Jones, Ian

AU - Levin, Scott

AU - Zhou, Chuan

AU - Gadd, Cynthia S.

AU - Aronsky, Dominik

PY - 2009/10

Y1 - 2009/10

N2 - Study objective: We apply a previously described tool to forecast emergency department (ED) crowding at multiple institutions and assess its generalizability for predicting the near-future waiting count, occupancy level, and boarding count. Methods: The ForecastED tool was validated with historical data from 5 institutions external to the development site. A sliding-window design separated the data for parameter estimation and forecast validation. Observations were sampled at consecutive 10-minute intervals during 12 months (n=52,560) at 4 sites and 10 months (n=44,064) at the fifth. Three outcome measures-the waiting count, occupancy level, and boarding count-were forecast 2, 4, 6, and 8 hours beyond each observation, and forecasts were compared with observed data at corresponding times. The reliability and calibration were measured following previously described methods. After linear calibration, the forecasting accuracy was measured with the median absolute error. Results: The tool was successfully used for 5 different sites. Its forecasts were more reliable, better calibrated, and more accurate at 2 hours than at 8 hours. The reliability and calibration of the tool were similar between the original development site and external sites; the boarding count was an exception, which was less reliable at 4 of 5 sites. Some variability in accuracy existed among institutions; when forecasting 4 hours into the future, the median absolute error of the waiting count ranged between 0.6 and 3.1 patients, the median absolute error of the occupancy level ranged between 9.0% and 14.5% of beds, and the median absolute error of the boarding count ranged between 0.9 and 2.8 patients. Conclusion: The ForecastED tool generated potentially useful forecasts of input and throughput measures of ED crowding at 5 external sites, without modifying the underlying assumptions. Noting the limitation that this was not a real-time validation, ongoing research will focus on integrating the tool with ED information systems.

AB - Study objective: We apply a previously described tool to forecast emergency department (ED) crowding at multiple institutions and assess its generalizability for predicting the near-future waiting count, occupancy level, and boarding count. Methods: The ForecastED tool was validated with historical data from 5 institutions external to the development site. A sliding-window design separated the data for parameter estimation and forecast validation. Observations were sampled at consecutive 10-minute intervals during 12 months (n=52,560) at 4 sites and 10 months (n=44,064) at the fifth. Three outcome measures-the waiting count, occupancy level, and boarding count-were forecast 2, 4, 6, and 8 hours beyond each observation, and forecasts were compared with observed data at corresponding times. The reliability and calibration were measured following previously described methods. After linear calibration, the forecasting accuracy was measured with the median absolute error. Results: The tool was successfully used for 5 different sites. Its forecasts were more reliable, better calibrated, and more accurate at 2 hours than at 8 hours. The reliability and calibration of the tool were similar between the original development site and external sites; the boarding count was an exception, which was less reliable at 4 of 5 sites. Some variability in accuracy existed among institutions; when forecasting 4 hours into the future, the median absolute error of the waiting count ranged between 0.6 and 3.1 patients, the median absolute error of the occupancy level ranged between 9.0% and 14.5% of beds, and the median absolute error of the boarding count ranged between 0.9 and 2.8 patients. Conclusion: The ForecastED tool generated potentially useful forecasts of input and throughput measures of ED crowding at 5 external sites, without modifying the underlying assumptions. Noting the limitation that this was not a real-time validation, ongoing research will focus on integrating the tool with ED information systems.

UR - http://www.scopus.com/inward/record.url?scp=70349159073&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70349159073&partnerID=8YFLogxK

U2 - 10.1016/j.annemergmed.2009.06.006

DO - 10.1016/j.annemergmed.2009.06.006

M3 - Article

C2 - 19716629

AN - SCOPUS:70349159073

VL - 54

JO - Annals of Emergency Medicine

JF - Annals of Emergency Medicine

SN - 0196-0644

IS - 4

ER -