Model performance metrics in assessing the value of adding intraoperative data for death prediction: Applications to noncardiac surgery

Victor J. Lei, Edward H. Kennedy, Thaibinh Luong, Xinwei Chen, Daniel E. Polsky, Kevin G. Volpp, Mark D. Neuman, John H. Holmes, Lee A. Fleisher, Amol S. Navathe

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We tested the value of adding data from the operating room to models predicting in-hospital death. We assessed model performance using two metrics, the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC), to illustrate the differences in information they convey in the setting of class imbalance. Data was collected on 74,147 patients who underwent major noncardiac surgery and 112 unique features were extracted from electronic health records. Sets of features were incrementally added to models using logistic regression, naïve Bayes, random forest, and gradient boosted machine methods. AUROC increased as more features were added, but changes were small for some modeling approaches. In contrast, AUPRC, which reflects positive predicted value, exhibited improvements across all models. Using AUPRC highlighted the added value of intraoperative data, not seen consistently with AUROC, and that with class imbalance AUPRC may serve as the more clinically relevant criterion.

Original languageEnglish (US)
Title of host publicationMEDINFO 2019
Subtitle of host publicationHealth and Wellbeing e-Networks for All - Proceedings of the 17th World Congress on Medical and Health Informatics
EditorsBrigitte Seroussi, Lucila Ohno-Machado, Lucila Ohno-Machado, Brigitte Seroussi
PublisherIOS Press
Pages223-227
Number of pages5
ISBN (Electronic)9781643680026
DOIs
StatePublished - Aug 21 2019
Externally publishedYes
Event17th World Congress on Medical and Health Informatics, MEDINFO 2019 - Lyon, France
Duration: Aug 25 2019Aug 30 2019

Publication series

NameStudies in Health Technology and Informatics
Volume264
ISSN (Print)0926-9630
ISSN (Electronic)1879-8365

Conference

Conference17th World Congress on Medical and Health Informatics, MEDINFO 2019
CountryFrance
CityLyon
Period8/25/198/30/19

Keywords

  • Hospital Mortality
  • Medical Informatics
  • Risk Assessment

ASJC Scopus subject areas

  • Biomedical Engineering
  • Health Informatics
  • Health Information Management

Fingerprint Dive into the research topics of 'Model performance metrics in assessing the value of adding intraoperative data for death prediction: Applications to noncardiac surgery'. Together they form a unique fingerprint.

  • Cite this

    Lei, V. J., Kennedy, E. H., Luong, T., Chen, X., Polsky, D. E., Volpp, K. G., Neuman, M. D., Holmes, J. H., Fleisher, L. A., & Navathe, A. S. (2019). Model performance metrics in assessing the value of adding intraoperative data for death prediction: Applications to noncardiac surgery. In B. Seroussi, L. Ohno-Machado, L. Ohno-Machado, & B. Seroussi (Eds.), MEDINFO 2019: Health and Wellbeing e-Networks for All - Proceedings of the 17th World Congress on Medical and Health Informatics (pp. 223-227). (Studies in Health Technology and Informatics; Vol. 264). IOS Press. https://doi.org/10.3233/SHTI190216