Regression coefficient–based scoring system should be used to assign weights to the risk index

Hemalkumar B. Mehta, Vinay Mehta, Cynthia J. Girman, Deepak Adhikari, Michael L. Johnson

Research output: Contribution to journalArticle

Abstract

Objective Some previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient–based and risk ratio–based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality. Study Design and Setting This retrospective cohort study included elderly people from the Clinical Practice Research Datalink. Cox proportional hazards regression models were constructed for time to 1-year mortality. Weights were assigned to 17 comorbidities using regression coefficient–based and risk ratio–based scoring systems. Different versions of CCS were compared using Akaike information criteria (AIC), McFadden's adjusted R2, and net reclassification improvement (NRI). Results Regression coefficient–based models (Beta, Beta10/integer, Beta/Schneeweiss, Beta/Sullivan) had lower AIC and higher R2 compared to risk ratio–based models (HR/Charlson, HR/Johnson). Regression coefficient–based CCS reclassified more number of people into the correct strata (NRI range, 9.02–10.04) compared to risk ratio–based CCS (NRI range, 8.14–8.22). Conclusion Previously developed risk scores contained an error in their construction adding ratios instead of multiplying them. Furthermore, as demonstrated here, adding ratios fail to even work adequately from a practical standpoint. CCS derived using regression coefficients performed slightly better than in fitting the data compared to risk ratio–based scoring systems. Researchers should use a regression coefficient–based scoring system to develop a risk index, which is theoretically correct.

Original languageEnglish (US)
Pages (from-to)22-28
Number of pages7
JournalJournal of Clinical Epidemiology
Volume79
DOIs
StatePublished - Nov 1 2016
Externally publishedYes

Fingerprint

Weights and Measures
Comorbidity
Mortality
Proportional Hazards Models
Cohort Studies
Retrospective Studies
Odds Ratio
Research Personnel
Research

Keywords

  • Charlson comorbidity score
  • Regression coefficient
  • Risk index
  • Risk ratio
  • Scoring algorithm
  • Scoring system

ASJC Scopus subject areas

  • Epidemiology

Cite this

Regression coefficient–based scoring system should be used to assign weights to the risk index. / Mehta, Hemalkumar B.; Mehta, Vinay; Girman, Cynthia J.; Adhikari, Deepak; Johnson, Michael L.

In: Journal of Clinical Epidemiology, Vol. 79, 01.11.2016, p. 22-28.

Research output: Contribution to journalArticle

Mehta, Hemalkumar B. ; Mehta, Vinay ; Girman, Cynthia J. ; Adhikari, Deepak ; Johnson, Michael L. / Regression coefficient–based scoring system should be used to assign weights to the risk index. In: Journal of Clinical Epidemiology. 2016 ; Vol. 79. pp. 22-28.
@article{401cb9652b27407182fe7fc6de99f1de,
title = "Regression coefficient–based scoring system should be used to assign weights to the risk index",
abstract = "Objective Some previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient–based and risk ratio–based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality. Study Design and Setting This retrospective cohort study included elderly people from the Clinical Practice Research Datalink. Cox proportional hazards regression models were constructed for time to 1-year mortality. Weights were assigned to 17 comorbidities using regression coefficient–based and risk ratio–based scoring systems. Different versions of CCS were compared using Akaike information criteria (AIC), McFadden's adjusted R2, and net reclassification improvement (NRI). Results Regression coefficient–based models (Beta, Beta10/integer, Beta/Schneeweiss, Beta/Sullivan) had lower AIC and higher R2 compared to risk ratio–based models (HR/Charlson, HR/Johnson). Regression coefficient–based CCS reclassified more number of people into the correct strata (NRI range, 9.02–10.04) compared to risk ratio–based CCS (NRI range, 8.14–8.22). Conclusion Previously developed risk scores contained an error in their construction adding ratios instead of multiplying them. Furthermore, as demonstrated here, adding ratios fail to even work adequately from a practical standpoint. CCS derived using regression coefficients performed slightly better than in fitting the data compared to risk ratio–based scoring systems. Researchers should use a regression coefficient–based scoring system to develop a risk index, which is theoretically correct.",
keywords = "Charlson comorbidity score, Regression coefficient, Risk index, Risk ratio, Scoring algorithm, Scoring system",
author = "Mehta, {Hemalkumar B.} and Vinay Mehta and Girman, {Cynthia J.} and Deepak Adhikari and Johnson, {Michael L.}",
year = "2016",
month = "11",
day = "1",
doi = "10.1016/j.jclinepi.2016.03.031",
language = "English (US)",
volume = "79",
pages = "22--28",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",

}

TY - JOUR

T1 - Regression coefficient–based scoring system should be used to assign weights to the risk index

AU - Mehta, Hemalkumar B.

AU - Mehta, Vinay

AU - Girman, Cynthia J.

AU - Adhikari, Deepak

AU - Johnson, Michael L.

PY - 2016/11/1

Y1 - 2016/11/1

N2 - Objective Some previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient–based and risk ratio–based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality. Study Design and Setting This retrospective cohort study included elderly people from the Clinical Practice Research Datalink. Cox proportional hazards regression models were constructed for time to 1-year mortality. Weights were assigned to 17 comorbidities using regression coefficient–based and risk ratio–based scoring systems. Different versions of CCS were compared using Akaike information criteria (AIC), McFadden's adjusted R2, and net reclassification improvement (NRI). Results Regression coefficient–based models (Beta, Beta10/integer, Beta/Schneeweiss, Beta/Sullivan) had lower AIC and higher R2 compared to risk ratio–based models (HR/Charlson, HR/Johnson). Regression coefficient–based CCS reclassified more number of people into the correct strata (NRI range, 9.02–10.04) compared to risk ratio–based CCS (NRI range, 8.14–8.22). Conclusion Previously developed risk scores contained an error in their construction adding ratios instead of multiplying them. Furthermore, as demonstrated here, adding ratios fail to even work adequately from a practical standpoint. CCS derived using regression coefficients performed slightly better than in fitting the data compared to risk ratio–based scoring systems. Researchers should use a regression coefficient–based scoring system to develop a risk index, which is theoretically correct.

AB - Objective Some previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient–based and risk ratio–based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality. Study Design and Setting This retrospective cohort study included elderly people from the Clinical Practice Research Datalink. Cox proportional hazards regression models were constructed for time to 1-year mortality. Weights were assigned to 17 comorbidities using regression coefficient–based and risk ratio–based scoring systems. Different versions of CCS were compared using Akaike information criteria (AIC), McFadden's adjusted R2, and net reclassification improvement (NRI). Results Regression coefficient–based models (Beta, Beta10/integer, Beta/Schneeweiss, Beta/Sullivan) had lower AIC and higher R2 compared to risk ratio–based models (HR/Charlson, HR/Johnson). Regression coefficient–based CCS reclassified more number of people into the correct strata (NRI range, 9.02–10.04) compared to risk ratio–based CCS (NRI range, 8.14–8.22). Conclusion Previously developed risk scores contained an error in their construction adding ratios instead of multiplying them. Furthermore, as demonstrated here, adding ratios fail to even work adequately from a practical standpoint. CCS derived using regression coefficients performed slightly better than in fitting the data compared to risk ratio–based scoring systems. Researchers should use a regression coefficient–based scoring system to develop a risk index, which is theoretically correct.

KW - Charlson comorbidity score

KW - Regression coefficient

KW - Risk index

KW - Risk ratio

KW - Scoring algorithm

KW - Scoring system

UR - http://www.scopus.com/inward/record.url?scp=84977492400&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84977492400&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2016.03.031

DO - 10.1016/j.jclinepi.2016.03.031

M3 - Article

C2 - 27181564

AN - SCOPUS:84977492400

VL - 79

SP - 22

EP - 28

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -