TY - JOUR
T1 - What are the real rates of postoperative complications
T2 - Elucidating inconsistencies between administrative and clinical data sources
AU - Koch, Colleen G.
AU - Li, Liang
AU - Hixson, Eric
AU - Tang, Anne
AU - Phillips, Shannon
AU - Henderson, J. Michael
PY - 2012/5/1
Y1 - 2012/5/1
N2 - Background: Comparison of quality outcomes generated from administrative and clinical datasets have shown inconsistencies. Understanding this is important because data designed to drive performance improvement are used for public reporting of performance. We examined administrative and clinical data and 2 clinical data sources in 4 surgical morbidity outcomes. Study Design: Patients who underwent operations between January 2009 and May 2010 had outcomes compared for postoperative hemorrhage, respiratory failure, deep vein thrombosis (DVT), and sepsis. Three data sources were examined: administrative (Agency for Healthcare Research and Quality [AHRQ] Patient Safety Indicators [PSIs]), a national clinical registry (National Surgical Quality Improvement Program [NSQIP]), and an institutional clinical registry (Cardiovascular Information Registry [CVIR]). Cohen's Kappa (K) coefficient was used as a measure of agreement between data sources. Results: For 4,583 patients common to AHRQ and NSQIP, concordance was poor for sepsis (K = 0.07) and hemorrhage (K = 0.14), moderate for respiratory failure (K = 0.30), and better concordance for DVT (K = 0.60). For 7,897 patients common to AHRQ and CVIR, concordance was poor for hemorrhage (K = 0.08), respiratory failure (K = 0.02), and sepsis (K = 0.16), and better for DVT (K = 0.55). For 886 patients common to NSQIP and CVIR, concordance was poor for sepsis (K = 0.054), moderate for hemorrhage (K = 0.27) and respiratory failure (K = 0.4), and better for DVT (K = 0.51). Conclusions: We demonstrate considerable discordance between data sources measuring the same postoperative events. The main contributor was difference in definitions, with additional contribution from data collection and management methods. Although any of these sources can be used for their original intent of performance improvement, this study emphasizes the shortcomings of using these sources for grading performance without standardizing definitions, data collection, and management.
AB - Background: Comparison of quality outcomes generated from administrative and clinical datasets have shown inconsistencies. Understanding this is important because data designed to drive performance improvement are used for public reporting of performance. We examined administrative and clinical data and 2 clinical data sources in 4 surgical morbidity outcomes. Study Design: Patients who underwent operations between January 2009 and May 2010 had outcomes compared for postoperative hemorrhage, respiratory failure, deep vein thrombosis (DVT), and sepsis. Three data sources were examined: administrative (Agency for Healthcare Research and Quality [AHRQ] Patient Safety Indicators [PSIs]), a national clinical registry (National Surgical Quality Improvement Program [NSQIP]), and an institutional clinical registry (Cardiovascular Information Registry [CVIR]). Cohen's Kappa (K) coefficient was used as a measure of agreement between data sources. Results: For 4,583 patients common to AHRQ and NSQIP, concordance was poor for sepsis (K = 0.07) and hemorrhage (K = 0.14), moderate for respiratory failure (K = 0.30), and better concordance for DVT (K = 0.60). For 7,897 patients common to AHRQ and CVIR, concordance was poor for hemorrhage (K = 0.08), respiratory failure (K = 0.02), and sepsis (K = 0.16), and better for DVT (K = 0.55). For 886 patients common to NSQIP and CVIR, concordance was poor for sepsis (K = 0.054), moderate for hemorrhage (K = 0.27) and respiratory failure (K = 0.4), and better for DVT (K = 0.51). Conclusions: We demonstrate considerable discordance between data sources measuring the same postoperative events. The main contributor was difference in definitions, with additional contribution from data collection and management methods. Although any of these sources can be used for their original intent of performance improvement, this study emphasizes the shortcomings of using these sources for grading performance without standardizing definitions, data collection, and management.
UR - http://www.scopus.com/inward/record.url?scp=84862812352&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84862812352&partnerID=8YFLogxK
U2 - 10.1016/j.jamcollsurg.2011.12.037
DO - 10.1016/j.jamcollsurg.2011.12.037
M3 - Article
C2 - 22421260
AN - SCOPUS:84862812352
VL - 214
SP - 798
EP - 805
JO - Surgery Gynecology and Obstetrics
JF - Surgery Gynecology and Obstetrics
SN - 1072-7515
IS - 5
ER -