TY - JOUR
T1 - Reporting and implementing interventions involving machine learning and artificial intelligence
AU - Bates, David W.
AU - Auerbach, Andrew
AU - Schulam, Peter
AU - Wright, Adam
AU - Saria, Suchi
N1 - Funding Information:
Disclosures: Dr. Bates reports grants from the Gordon and Betty Moore Foundation during the conduct of the study, and grants from EarlySense and IBM Watson Health and personal fees from EarlySense CDI (Negev), Ltd, ValeraHealth, Clew, and MDClone outside the submitted work. Dr. Schulam reports personal fees from Bayesian Health outside the submitted work. Dr. Saria reports grants from the Gordon and Betty Moore Foundation, National Science Foundation, National Institutes of Health, Defense Advanced Research Projects Agency (DARPA), and the American Heart Association outside the submitted work. Dr. Saria is a founder of and holds equity in Bayesian Health. This arrangement has been reviewed and approved by the Johns Hopkins University in accordance with its conflict of interest policies. She is the scientific advisory board member for PatientPing. She has received honoraria for talks from a number of biotechnology, research, and health technology companies. She has no direct ties to the companies mentioned in this article. Authors not named here have disclosed no conflicts of interest. Disclosures can also be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do ?msNum=M19-0872.
Publisher Copyright:
© 2020 American College of Physicians. All rights reserved.
PY - 2020/6/2
Y1 - 2020/6/2
N2 - Increasingly, interventions aimed at improving care are likely to use such technologies as machine learning and artificial intelligence. However, health care has been relatively late to adopt them. This article provides clinical examples in which machine learning and artificial intelligence are already in use in health care and appear to deliver benefit. Three key bottlenecks toward increasing the pace of diffusion and adoption are methodological issues in evaluation of artificial intelligence-based interventions, reporting standards to enable assessment of model performance, and issues that need to be addressed for an institution to adopt these interventions. Methodological best practices will include external validation, ideally at a different site; use of proactive learning algorithms to correct for site-specific biases and increase robustness as algorithms are deployed across multiple sites; addressing subgroup performance; and communicating to providers the uncertainty of predictions. Regarding reporting, especially important issues are the extent to which implementing standardized approaches for introducing clinical decision support has been followed, describing the data sources, reporting on data assumptions, and addressing biases. Although most health care organizations in the United States have adopted electronic health records, they may be ill prepared to adopt machine learning and artificial intelligence. Several steps can enable this: Preparing data, developing tools to get suggestions to clinicians in useful ways, and getting clinicians engaged in the process. Open challenges and the role of regulation in this area are briefly discussed. Although these techniques have enormous potential to improve care and personalize recommendations for individuals, the hype regarding them is tremendous. Organizations will need to approach this domain carefully with knowledgeable partners to obtain the hoped-for benefits and avoid failures.
AB - Increasingly, interventions aimed at improving care are likely to use such technologies as machine learning and artificial intelligence. However, health care has been relatively late to adopt them. This article provides clinical examples in which machine learning and artificial intelligence are already in use in health care and appear to deliver benefit. Three key bottlenecks toward increasing the pace of diffusion and adoption are methodological issues in evaluation of artificial intelligence-based interventions, reporting standards to enable assessment of model performance, and issues that need to be addressed for an institution to adopt these interventions. Methodological best practices will include external validation, ideally at a different site; use of proactive learning algorithms to correct for site-specific biases and increase robustness as algorithms are deployed across multiple sites; addressing subgroup performance; and communicating to providers the uncertainty of predictions. Regarding reporting, especially important issues are the extent to which implementing standardized approaches for introducing clinical decision support has been followed, describing the data sources, reporting on data assumptions, and addressing biases. Although most health care organizations in the United States have adopted electronic health records, they may be ill prepared to adopt machine learning and artificial intelligence. Several steps can enable this: Preparing data, developing tools to get suggestions to clinicians in useful ways, and getting clinicians engaged in the process. Open challenges and the role of regulation in this area are briefly discussed. Although these techniques have enormous potential to improve care and personalize recommendations for individuals, the hype regarding them is tremendous. Organizations will need to approach this domain carefully with knowledgeable partners to obtain the hoped-for benefits and avoid failures.
UR - http://www.scopus.com/inward/record.url?scp=85085854660&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85085854660&partnerID=8YFLogxK
U2 - 10.7326/M19-0872
DO - 10.7326/M19-0872
M3 - Article
C2 - 32479180
AN - SCOPUS:85085854660
SN - 0003-4819
VL - 172
SP - S137-S144
JO - Annals of internal medicine
JF - Annals of internal medicine
IS - 11
ER -