Predicting complications of major head and neck oncological surgery: An evaluation of the ACS NSQIP surgical risk calculator

Peter Vosler, Mario Orsini, Danny J. Enepekides, Kevin M. Higgins

Research output: Contribution to journalArticle

Abstract

Background: The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) universal surgical risk calculator is an online tool intended to improve the informed consent process and surgical decision-making. The risk calculator uses a database of information from 585 hospitals to predict a patient's risk of developing specific postoperative outcomes. Methods: Patient records at a major Canadian tertiary care referral center between July 2015 and March 2017 were reviewed for surgical cases including one of six major head and neck oncologic surgeries: total thyroidectomy, total laryngectomy, hemiglossectomy, partial glossectomy, laryngopharyngectomy, and composite resection. Preoperative information for 107 patients was entered into the risk calculator and compared to observed postoperative outcomes. Statistical analysis of the risk calculator was completed for the entire study population, for stratification by procedure, and by utilization of microvascular reconstruction. Accuracy was assessed using the ratio of predicted to observed outcomes, Receiver Operating Characteristics (ROC), Brier score, and the Wilcoxon signed-ranked test. Results: The risk calculator accurately predicted the incidences for 11 of 12 outcomes for patients that did not undergo free flap reconstruction (NFF group), but was less accurate for patients that underwent free flap reconstruction (FF group). Length of stay (LOS) analysis showed similar results, with predicted and observed LOS statistically different in the overall population and FF group analyses (p = 0.001 for both), but not for the NFF group analysis (p = 0.764). All outcomes in the NFF group, when analyzed for calibration, met the threshold value (Brier scores < 0.09). Risk predictions for 8 of 12, and 10 of 12 outcomes were adequately calibrated in the FF group and the overall study population, respectively. Analyses by procedure were excellent, with the risk calculator showing adequate calibration for 7 of 8 procedural categories and adequate discrimination for all calculable categories (6 of 6). Conclusion: The NSQIP-RC demonstrated efficacy for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction. The predictive value of the metric can be improved by inclusion of several factors important for risk stratification in head and neck oncology.

Original languageEnglish (US)
Article number21
JournalJournal of Otolaryngology - Head and Neck Surgery
Volume47
Issue number1
DOIs
StatePublished - Mar 22 2018
Externally publishedYes

Fingerprint

Quality Improvement
Neck
Head
Free Tissue Flaps
Tertiary Care Centers
Calibration
Length of Stay
Glossectomy
Laryngectomy
Surgeons
Thyroidectomy
Informed Consent
Population Groups
ROC Curve
Population
Decision Making
Databases
Incidence

Keywords

  • Brier score
  • Cancer
  • Free flap reconstruction
  • Head and neck
  • Microvascular reconstruction
  • National surgical quality improvement program risk calculator
  • Outcome measures
  • Risk assessment
  • Surgical complications

ASJC Scopus subject areas

  • Surgery
  • Otorhinolaryngology

Cite this

Predicting complications of major head and neck oncological surgery : An evaluation of the ACS NSQIP surgical risk calculator. / Vosler, Peter; Orsini, Mario; Enepekides, Danny J.; Higgins, Kevin M.

In: Journal of Otolaryngology - Head and Neck Surgery, Vol. 47, No. 1, 21, 22.03.2018.

Research output: Contribution to journalArticle

@article{200845786b0b47faa644c833344eec34,
title = "Predicting complications of major head and neck oncological surgery: An evaluation of the ACS NSQIP surgical risk calculator",
abstract = "Background: The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) universal surgical risk calculator is an online tool intended to improve the informed consent process and surgical decision-making. The risk calculator uses a database of information from 585 hospitals to predict a patient's risk of developing specific postoperative outcomes. Methods: Patient records at a major Canadian tertiary care referral center between July 2015 and March 2017 were reviewed for surgical cases including one of six major head and neck oncologic surgeries: total thyroidectomy, total laryngectomy, hemiglossectomy, partial glossectomy, laryngopharyngectomy, and composite resection. Preoperative information for 107 patients was entered into the risk calculator and compared to observed postoperative outcomes. Statistical analysis of the risk calculator was completed for the entire study population, for stratification by procedure, and by utilization of microvascular reconstruction. Accuracy was assessed using the ratio of predicted to observed outcomes, Receiver Operating Characteristics (ROC), Brier score, and the Wilcoxon signed-ranked test. Results: The risk calculator accurately predicted the incidences for 11 of 12 outcomes for patients that did not undergo free flap reconstruction (NFF group), but was less accurate for patients that underwent free flap reconstruction (FF group). Length of stay (LOS) analysis showed similar results, with predicted and observed LOS statistically different in the overall population and FF group analyses (p = 0.001 for both), but not for the NFF group analysis (p = 0.764). All outcomes in the NFF group, when analyzed for calibration, met the threshold value (Brier scores < 0.09). Risk predictions for 8 of 12, and 10 of 12 outcomes were adequately calibrated in the FF group and the overall study population, respectively. Analyses by procedure were excellent, with the risk calculator showing adequate calibration for 7 of 8 procedural categories and adequate discrimination for all calculable categories (6 of 6). Conclusion: The NSQIP-RC demonstrated efficacy for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction. The predictive value of the metric can be improved by inclusion of several factors important for risk stratification in head and neck oncology.",
keywords = "Brier score, Cancer, Free flap reconstruction, Head and neck, Microvascular reconstruction, National surgical quality improvement program risk calculator, Outcome measures, Risk assessment, Surgical complications",
author = "Peter Vosler and Mario Orsini and Enepekides, {Danny J.} and Higgins, {Kevin M.}",
year = "2018",
month = "3",
day = "22",
doi = "10.1186/s40463-018-0269-8",
language = "English (US)",
volume = "47",
journal = "Journal of Otolaryngology - Head and Neck Surgery",
issn = "1916-0208",
publisher = "Decker Publishing",
number = "1",

}

TY - JOUR

T1 - Predicting complications of major head and neck oncological surgery

T2 - An evaluation of the ACS NSQIP surgical risk calculator

AU - Vosler, Peter

AU - Orsini, Mario

AU - Enepekides, Danny J.

AU - Higgins, Kevin M.

PY - 2018/3/22

Y1 - 2018/3/22

N2 - Background: The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) universal surgical risk calculator is an online tool intended to improve the informed consent process and surgical decision-making. The risk calculator uses a database of information from 585 hospitals to predict a patient's risk of developing specific postoperative outcomes. Methods: Patient records at a major Canadian tertiary care referral center between July 2015 and March 2017 were reviewed for surgical cases including one of six major head and neck oncologic surgeries: total thyroidectomy, total laryngectomy, hemiglossectomy, partial glossectomy, laryngopharyngectomy, and composite resection. Preoperative information for 107 patients was entered into the risk calculator and compared to observed postoperative outcomes. Statistical analysis of the risk calculator was completed for the entire study population, for stratification by procedure, and by utilization of microvascular reconstruction. Accuracy was assessed using the ratio of predicted to observed outcomes, Receiver Operating Characteristics (ROC), Brier score, and the Wilcoxon signed-ranked test. Results: The risk calculator accurately predicted the incidences for 11 of 12 outcomes for patients that did not undergo free flap reconstruction (NFF group), but was less accurate for patients that underwent free flap reconstruction (FF group). Length of stay (LOS) analysis showed similar results, with predicted and observed LOS statistically different in the overall population and FF group analyses (p = 0.001 for both), but not for the NFF group analysis (p = 0.764). All outcomes in the NFF group, when analyzed for calibration, met the threshold value (Brier scores < 0.09). Risk predictions for 8 of 12, and 10 of 12 outcomes were adequately calibrated in the FF group and the overall study population, respectively. Analyses by procedure were excellent, with the risk calculator showing adequate calibration for 7 of 8 procedural categories and adequate discrimination for all calculable categories (6 of 6). Conclusion: The NSQIP-RC demonstrated efficacy for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction. The predictive value of the metric can be improved by inclusion of several factors important for risk stratification in head and neck oncology.

AB - Background: The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) universal surgical risk calculator is an online tool intended to improve the informed consent process and surgical decision-making. The risk calculator uses a database of information from 585 hospitals to predict a patient's risk of developing specific postoperative outcomes. Methods: Patient records at a major Canadian tertiary care referral center between July 2015 and March 2017 were reviewed for surgical cases including one of six major head and neck oncologic surgeries: total thyroidectomy, total laryngectomy, hemiglossectomy, partial glossectomy, laryngopharyngectomy, and composite resection. Preoperative information for 107 patients was entered into the risk calculator and compared to observed postoperative outcomes. Statistical analysis of the risk calculator was completed for the entire study population, for stratification by procedure, and by utilization of microvascular reconstruction. Accuracy was assessed using the ratio of predicted to observed outcomes, Receiver Operating Characteristics (ROC), Brier score, and the Wilcoxon signed-ranked test. Results: The risk calculator accurately predicted the incidences for 11 of 12 outcomes for patients that did not undergo free flap reconstruction (NFF group), but was less accurate for patients that underwent free flap reconstruction (FF group). Length of stay (LOS) analysis showed similar results, with predicted and observed LOS statistically different in the overall population and FF group analyses (p = 0.001 for both), but not for the NFF group analysis (p = 0.764). All outcomes in the NFF group, when analyzed for calibration, met the threshold value (Brier scores < 0.09). Risk predictions for 8 of 12, and 10 of 12 outcomes were adequately calibrated in the FF group and the overall study population, respectively. Analyses by procedure were excellent, with the risk calculator showing adequate calibration for 7 of 8 procedural categories and adequate discrimination for all calculable categories (6 of 6). Conclusion: The NSQIP-RC demonstrated efficacy for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction. The predictive value of the metric can be improved by inclusion of several factors important for risk stratification in head and neck oncology.

KW - Brier score

KW - Cancer

KW - Free flap reconstruction

KW - Head and neck

KW - Microvascular reconstruction

KW - National surgical quality improvement program risk calculator

KW - Outcome measures

KW - Risk assessment

KW - Surgical complications

UR - http://www.scopus.com/inward/record.url?scp=85044384463&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044384463&partnerID=8YFLogxK

U2 - 10.1186/s40463-018-0269-8

DO - 10.1186/s40463-018-0269-8

M3 - Article

C2 - 29566750

AN - SCOPUS:85044384463

VL - 47

JO - Journal of Otolaryngology - Head and Neck Surgery

JF - Journal of Otolaryngology - Head and Neck Surgery

SN - 1916-0208

IS - 1

M1 - 21

ER -