Utility of prediction model score: A proposed tool to standardize the performance and generalizability of clinical predictive models based on systematic review

Jeff Ehresman, Daniel Lubelski, Zach Pennington, Bethany Hung, A. Karim Ahmed, Tej D. Azad, Kurt Lehner, James Feghali, Zorica Buser, James Harrop, Jefferson Wilson, Shekar Kurpad, Zoher Ghogawala, Daniel M. Sciubba

Research output: Contribution to journalArticlepeer-review

Abstract

OBJECTIVE The objective of this study was to evaluate the characteristics and performance of current prediction models in the fields of spine metastasis and degenerative spine disease to create a scoring system that allows direct comparison of the prediction models. METHODS A systematic search of PubMed and Embase was performed to identify relevant studies that included either the proposal of a prediction model or an external validation of a previously proposed prediction model with 1-year outcomes. Characteristics of the original study and discriminative performance of external validations were then assigned points based on thresholds from the overall cohort. RESULTS Nine prediction models were included in the spine metastasis category, while 6 prediction models were included in the degenerative spine category. After assigning the proposed utility of prediction model score to the spine metastasis prediction models, only 1 reached the grade of excellent, while 2 were graded as good, 3 as fair, and 3 as poor. Of the 6 included degenerative spine models, 1 reached the excellent grade, while 3 studies were graded as good, 1 as fair, and 1 as poor. CONCLUSIONS As interest in utilizing predictive analytics in spine surgery increases, there is a concomitant increase in the number of published prediction models that differ in methodology and performance. Prior to applying these models to patient care, these models must be evaluated. To begin addressing this issue, the authors proposed a grading system that compares these models based on various metrics related to their original design as well as internal and external validation. Ultimately, this may hopefully aid clinicians in determining the relative validity and usability of a given model.

Original languageEnglish (US)
Pages (from-to)779-787
Number of pages9
JournalJournal of Neurosurgery: Spine
Volume34
Issue number5
DOIs
StatePublished - May 2021

Keywords

  • Degenerative
  • Diagnostic technique
  • Generalizability
  • Oncology
  • Performance
  • Prediction model
  • Predictive analytics
  • Prognostic
  • Spine metastasis
  • Survival

ASJC Scopus subject areas

  • Surgery
  • Neurology
  • Clinical Neurology

Fingerprint

Dive into the research topics of 'Utility of prediction model score: A proposed tool to standardize the performance and generalizability of clinical predictive models based on systematic review'. Together they form a unique fingerprint.

Cite this