A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models

H. Echo Wang, Matthew Landers, Roy Adams, Adarsh Subbaswamy, Hadi Kharrazi, Darrell J. Gaskin, Suchi Saria

Research output: Contribution to journalArticlepeer-review

Abstract

Objective: Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model's potential to introduce bias. Materials and Methods: Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist. Results: We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern. Discussion: Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed. Conclusion: The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

Original languageEnglish (US)
Pages (from-to)1323-1333
Number of pages11
JournalJournal of the American Medical Informatics Association
Volume29
Issue number8
DOIs
StatePublished - Aug 1 2022

Keywords

  • bias
  • clinical decision-making
  • health care disparity
  • hospital readmission
  • predictive model

ASJC Scopus subject areas

  • Health Informatics

Fingerprint

Dive into the research topics of 'A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models'. Together they form a unique fingerprint.

Cite this