Identification of patients expected to benefit from electronic alerts for acute kidney injury

Aditya Biswas, Chirag Parikh, Harold I. Feldman, Amit X. Garg, Stephen Latham, Haiqun Lin, Paul M. Palevsky, Ugochukwu Ugwuowo, F. Perry Wilson

Research output: Contribution to journalArticle

Abstract

Background and objectives Electronic alerts for heterogenous conditions such as AKI may not provide benefit for all eligible patients and can lead to alert fatigue, suggesting that personalized alert targeting may be useful. Uplift-based alert targeting may be superior to purely prognostic-targeting of interventions because uplift models assess marginal treatment effect rather than likelihood of outcome. Design, setting, participants, & measurements This is a secondary analysis of a clinical trial of 2278 adult patients with AKI randomized to an automated, electronic alert system versus usual care. We used three uplift algorithms and one purely prognostic algorithm, trained in 70% of the data, and evaluated the effect of targeting alerts to patients with higher scores in the held-out 30% of the data. The performance of the targeting strategy was assessed as the interaction between the model prediction of likelihood to benefit from alerts and randomization status. The outcome of interest was maximum relative change in creatinine from the time of randomization to 3 days after randomization. Results The three uplift score algorithms all gave rise to a significant interaction term, suggesting that a strategy of targeting individuals with higher uplift scores would lead to a beneficial effect of AKI alerting, in contrast to the null effect seen in the overall study. The prognostic model did not successfully stratify patients with regards to benefit of the intervention. Among individuals in the high uplift group, alerting was associated with a median reduction in change in creatinine of 25.3% (P=0.03). In the low uplift group, alerting was associated with a median increase in change in creatinine of +5.3% (P=0.005). Older individuals, women, and those with a lower randomization creatinine were more likely to receive high uplift scores, suggesting that alerts may benefit those with more slowly developing AKI. Conclusions Uplift modeling, which accounts for treatment effect, can successfully target electronic alerts for AKI to those most likely to benefit, whereas purely prognostic targeting cannot.

Original languageEnglish (US)
Pages (from-to)842-849
Number of pages8
JournalClinical Journal of the American Society of Nephrology
Volume13
Issue number6
DOIs
StatePublished - Jun 7 2018

Fingerprint

Random Allocation
Acute Kidney Injury
Creatinine
Fatigue
Clinical Trials
Therapeutics

ASJC Scopus subject areas

  • Epidemiology
  • Critical Care and Intensive Care Medicine
  • Nephrology
  • Transplantation

Cite this

Identification of patients expected to benefit from electronic alerts for acute kidney injury. / Biswas, Aditya; Parikh, Chirag; Feldman, Harold I.; Garg, Amit X.; Latham, Stephen; Lin, Haiqun; Palevsky, Paul M.; Ugwuowo, Ugochukwu; Wilson, F. Perry.

In: Clinical Journal of the American Society of Nephrology, Vol. 13, No. 6, 07.06.2018, p. 842-849.

Research output: Contribution to journalArticle

Biswas, A, Parikh, C, Feldman, HI, Garg, AX, Latham, S, Lin, H, Palevsky, PM, Ugwuowo, U & Wilson, FP 2018, 'Identification of patients expected to benefit from electronic alerts for acute kidney injury', Clinical Journal of the American Society of Nephrology, vol. 13, no. 6, pp. 842-849. https://doi.org/10.2215/CJN.13351217
Biswas, Aditya ; Parikh, Chirag ; Feldman, Harold I. ; Garg, Amit X. ; Latham, Stephen ; Lin, Haiqun ; Palevsky, Paul M. ; Ugwuowo, Ugochukwu ; Wilson, F. Perry. / Identification of patients expected to benefit from electronic alerts for acute kidney injury. In: Clinical Journal of the American Society of Nephrology. 2018 ; Vol. 13, No. 6. pp. 842-849.
@article{8f167896e0c144dda4d903eff341922c,
title = "Identification of patients expected to benefit from electronic alerts for acute kidney injury",
abstract = "Background and objectives Electronic alerts for heterogenous conditions such as AKI may not provide benefit for all eligible patients and can lead to alert fatigue, suggesting that personalized alert targeting may be useful. Uplift-based alert targeting may be superior to purely prognostic-targeting of interventions because uplift models assess marginal treatment effect rather than likelihood of outcome. Design, setting, participants, & measurements This is a secondary analysis of a clinical trial of 2278 adult patients with AKI randomized to an automated, electronic alert system versus usual care. We used three uplift algorithms and one purely prognostic algorithm, trained in 70{\%} of the data, and evaluated the effect of targeting alerts to patients with higher scores in the held-out 30{\%} of the data. The performance of the targeting strategy was assessed as the interaction between the model prediction of likelihood to benefit from alerts and randomization status. The outcome of interest was maximum relative change in creatinine from the time of randomization to 3 days after randomization. Results The three uplift score algorithms all gave rise to a significant interaction term, suggesting that a strategy of targeting individuals with higher uplift scores would lead to a beneficial effect of AKI alerting, in contrast to the null effect seen in the overall study. The prognostic model did not successfully stratify patients with regards to benefit of the intervention. Among individuals in the high uplift group, alerting was associated with a median reduction in change in creatinine of 25.3{\%} (P=0.03). In the low uplift group, alerting was associated with a median increase in change in creatinine of +5.3{\%} (P=0.005). Older individuals, women, and those with a lower randomization creatinine were more likely to receive high uplift scores, suggesting that alerts may benefit those with more slowly developing AKI. Conclusions Uplift modeling, which accounts for treatment effect, can successfully target electronic alerts for AKI to those most likely to benefit, whereas purely prognostic targeting cannot.",
author = "Aditya Biswas and Chirag Parikh and Feldman, {Harold I.} and Garg, {Amit X.} and Stephen Latham and Haiqun Lin and Palevsky, {Paul M.} and Ugochukwu Ugwuowo and Wilson, {F. Perry}",
year = "2018",
month = "6",
day = "7",
doi = "10.2215/CJN.13351217",
language = "English (US)",
volume = "13",
pages = "842--849",
journal = "Clinical journal of the American Society of Nephrology : CJASN",
issn = "1555-9041",
publisher = "American Society of Nephrology",
number = "6",

}

TY - JOUR

T1 - Identification of patients expected to benefit from electronic alerts for acute kidney injury

AU - Biswas, Aditya

AU - Parikh, Chirag

AU - Feldman, Harold I.

AU - Garg, Amit X.

AU - Latham, Stephen

AU - Lin, Haiqun

AU - Palevsky, Paul M.

AU - Ugwuowo, Ugochukwu

AU - Wilson, F. Perry

PY - 2018/6/7

Y1 - 2018/6/7

N2 - Background and objectives Electronic alerts for heterogenous conditions such as AKI may not provide benefit for all eligible patients and can lead to alert fatigue, suggesting that personalized alert targeting may be useful. Uplift-based alert targeting may be superior to purely prognostic-targeting of interventions because uplift models assess marginal treatment effect rather than likelihood of outcome. Design, setting, participants, & measurements This is a secondary analysis of a clinical trial of 2278 adult patients with AKI randomized to an automated, electronic alert system versus usual care. We used three uplift algorithms and one purely prognostic algorithm, trained in 70% of the data, and evaluated the effect of targeting alerts to patients with higher scores in the held-out 30% of the data. The performance of the targeting strategy was assessed as the interaction between the model prediction of likelihood to benefit from alerts and randomization status. The outcome of interest was maximum relative change in creatinine from the time of randomization to 3 days after randomization. Results The three uplift score algorithms all gave rise to a significant interaction term, suggesting that a strategy of targeting individuals with higher uplift scores would lead to a beneficial effect of AKI alerting, in contrast to the null effect seen in the overall study. The prognostic model did not successfully stratify patients with regards to benefit of the intervention. Among individuals in the high uplift group, alerting was associated with a median reduction in change in creatinine of 25.3% (P=0.03). In the low uplift group, alerting was associated with a median increase in change in creatinine of +5.3% (P=0.005). Older individuals, women, and those with a lower randomization creatinine were more likely to receive high uplift scores, suggesting that alerts may benefit those with more slowly developing AKI. Conclusions Uplift modeling, which accounts for treatment effect, can successfully target electronic alerts for AKI to those most likely to benefit, whereas purely prognostic targeting cannot.

AB - Background and objectives Electronic alerts for heterogenous conditions such as AKI may not provide benefit for all eligible patients and can lead to alert fatigue, suggesting that personalized alert targeting may be useful. Uplift-based alert targeting may be superior to purely prognostic-targeting of interventions because uplift models assess marginal treatment effect rather than likelihood of outcome. Design, setting, participants, & measurements This is a secondary analysis of a clinical trial of 2278 adult patients with AKI randomized to an automated, electronic alert system versus usual care. We used three uplift algorithms and one purely prognostic algorithm, trained in 70% of the data, and evaluated the effect of targeting alerts to patients with higher scores in the held-out 30% of the data. The performance of the targeting strategy was assessed as the interaction between the model prediction of likelihood to benefit from alerts and randomization status. The outcome of interest was maximum relative change in creatinine from the time of randomization to 3 days after randomization. Results The three uplift score algorithms all gave rise to a significant interaction term, suggesting that a strategy of targeting individuals with higher uplift scores would lead to a beneficial effect of AKI alerting, in contrast to the null effect seen in the overall study. The prognostic model did not successfully stratify patients with regards to benefit of the intervention. Among individuals in the high uplift group, alerting was associated with a median reduction in change in creatinine of 25.3% (P=0.03). In the low uplift group, alerting was associated with a median increase in change in creatinine of +5.3% (P=0.005). Older individuals, women, and those with a lower randomization creatinine were more likely to receive high uplift scores, suggesting that alerts may benefit those with more slowly developing AKI. Conclusions Uplift modeling, which accounts for treatment effect, can successfully target electronic alerts for AKI to those most likely to benefit, whereas purely prognostic targeting cannot.

UR - http://www.scopus.com/inward/record.url?scp=85048297422&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048297422&partnerID=8YFLogxK

U2 - 10.2215/CJN.13351217

DO - 10.2215/CJN.13351217

M3 - Article

C2 - 29599299

AN - SCOPUS:85048297422

VL - 13

SP - 842

EP - 849

JO - Clinical journal of the American Society of Nephrology : CJASN

JF - Clinical journal of the American Society of Nephrology : CJASN

SN - 1555-9041

IS - 6

ER -