Estimating a composite measure of hospital quality from the hospital compare database: Differences when using a bayesian hierarchical latent variable model versus denominator-based weights

Michael Shwartz, Justin Ren, Erol A. Peköz, Xin Wang, Alan B. Cohen, Joseph D. Restuccia

Research output: Contribution to journalArticle

Abstract

BACKGROUND: A single composite measure calculated from individual quality indicators (QIs) is a useful measure of hospital performance and can be justified conceptually even when the indicators are not highly correlated with one another. OBJECTIVE: To compare 2 basic approaches for calculating a composite measure: an extension of the most widely-used approach, which weights individual indicators based on the number of people eligible for the indicator (referred to as denominator-based weights, DBWs), and a Bayesian hierarchical latent variable model (BLVM). METHODS: Using data for 15 QIs from 3275 hospitals in the Hospital Compare database, we calculated hospital ranks using several versions of DBWs and 2 BLVMs. Estimates in 1 BLVM were driven by differences in variances of the QIs (BLVM1) and estimates in the other by differences in the signal-to-noise ratios of the QIs (BLVM2). RESULTS: There was a high correlation in ranks among all of the DBW approaches and between those approaches and BLVM1. However, a high correlation does not necessarily mean that the same hospitals were ranked in the top or bottom quality deciles. In general, large hospitals were ranked in higher quality deciles by all of the approaches, though the effect was most apparent using BLVM2. CONCLUSIONS: Both conceptually and practically, hospital-specific DBWs are a reasonable approach for calculating a composite measure. However, this approach fails to take into account differences in the reliability of estimates from hospitals of different sizes, a big advantage of the Bayesian models.

Original languageEnglish (US)
Pages (from-to)778-785
Number of pages8
JournalMedical Care
Volume46
Issue number8
DOIs
StatePublished - Aug 2008
Externally publishedYes

Fingerprint

Databases
Weights and Measures
Health Facility Size
Signal-To-Noise Ratio
General Hospitals
performance

Keywords

  • Bayesian inference
  • Quality measurement
  • Quality performance

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health
  • Nursing(all)
  • Health(social science)
  • Health Professions(all)

Cite this

Estimating a composite measure of hospital quality from the hospital compare database : Differences when using a bayesian hierarchical latent variable model versus denominator-based weights. / Shwartz, Michael; Ren, Justin; Peköz, Erol A.; Wang, Xin; Cohen, Alan B.; Restuccia, Joseph D.

In: Medical Care, Vol. 46, No. 8, 08.2008, p. 778-785.

Research output: Contribution to journalArticle

Shwartz, Michael ; Ren, Justin ; Peköz, Erol A. ; Wang, Xin ; Cohen, Alan B. ; Restuccia, Joseph D. / Estimating a composite measure of hospital quality from the hospital compare database : Differences when using a bayesian hierarchical latent variable model versus denominator-based weights. In: Medical Care. 2008 ; Vol. 46, No. 8. pp. 778-785.
@article{b749db0df88e40cf92ed610205c51e57,
title = "Estimating a composite measure of hospital quality from the hospital compare database: Differences when using a bayesian hierarchical latent variable model versus denominator-based weights",
abstract = "BACKGROUND: A single composite measure calculated from individual quality indicators (QIs) is a useful measure of hospital performance and can be justified conceptually even when the indicators are not highly correlated with one another. OBJECTIVE: To compare 2 basic approaches for calculating a composite measure: an extension of the most widely-used approach, which weights individual indicators based on the number of people eligible for the indicator (referred to as denominator-based weights, DBWs), and a Bayesian hierarchical latent variable model (BLVM). METHODS: Using data for 15 QIs from 3275 hospitals in the Hospital Compare database, we calculated hospital ranks using several versions of DBWs and 2 BLVMs. Estimates in 1 BLVM were driven by differences in variances of the QIs (BLVM1) and estimates in the other by differences in the signal-to-noise ratios of the QIs (BLVM2). RESULTS: There was a high correlation in ranks among all of the DBW approaches and between those approaches and BLVM1. However, a high correlation does not necessarily mean that the same hospitals were ranked in the top or bottom quality deciles. In general, large hospitals were ranked in higher quality deciles by all of the approaches, though the effect was most apparent using BLVM2. CONCLUSIONS: Both conceptually and practically, hospital-specific DBWs are a reasonable approach for calculating a composite measure. However, this approach fails to take into account differences in the reliability of estimates from hospitals of different sizes, a big advantage of the Bayesian models.",
keywords = "Bayesian inference, Quality measurement, Quality performance",
author = "Michael Shwartz and Justin Ren and Pek{\"o}z, {Erol A.} and Xin Wang and Cohen, {Alan B.} and Restuccia, {Joseph D.}",
year = "2008",
month = "8",
doi = "10.1097/MLR.0b013e31817893dc",
language = "English (US)",
volume = "46",
pages = "778--785",
journal = "Medical Care",
issn = "0025-7079",
publisher = "Lippincott Williams and Wilkins",
number = "8",

}

TY - JOUR

T1 - Estimating a composite measure of hospital quality from the hospital compare database

T2 - Differences when using a bayesian hierarchical latent variable model versus denominator-based weights

AU - Shwartz, Michael

AU - Ren, Justin

AU - Peköz, Erol A.

AU - Wang, Xin

AU - Cohen, Alan B.

AU - Restuccia, Joseph D.

PY - 2008/8

Y1 - 2008/8

N2 - BACKGROUND: A single composite measure calculated from individual quality indicators (QIs) is a useful measure of hospital performance and can be justified conceptually even when the indicators are not highly correlated with one another. OBJECTIVE: To compare 2 basic approaches for calculating a composite measure: an extension of the most widely-used approach, which weights individual indicators based on the number of people eligible for the indicator (referred to as denominator-based weights, DBWs), and a Bayesian hierarchical latent variable model (BLVM). METHODS: Using data for 15 QIs from 3275 hospitals in the Hospital Compare database, we calculated hospital ranks using several versions of DBWs and 2 BLVMs. Estimates in 1 BLVM were driven by differences in variances of the QIs (BLVM1) and estimates in the other by differences in the signal-to-noise ratios of the QIs (BLVM2). RESULTS: There was a high correlation in ranks among all of the DBW approaches and between those approaches and BLVM1. However, a high correlation does not necessarily mean that the same hospitals were ranked in the top or bottom quality deciles. In general, large hospitals were ranked in higher quality deciles by all of the approaches, though the effect was most apparent using BLVM2. CONCLUSIONS: Both conceptually and practically, hospital-specific DBWs are a reasonable approach for calculating a composite measure. However, this approach fails to take into account differences in the reliability of estimates from hospitals of different sizes, a big advantage of the Bayesian models.

AB - BACKGROUND: A single composite measure calculated from individual quality indicators (QIs) is a useful measure of hospital performance and can be justified conceptually even when the indicators are not highly correlated with one another. OBJECTIVE: To compare 2 basic approaches for calculating a composite measure: an extension of the most widely-used approach, which weights individual indicators based on the number of people eligible for the indicator (referred to as denominator-based weights, DBWs), and a Bayesian hierarchical latent variable model (BLVM). METHODS: Using data for 15 QIs from 3275 hospitals in the Hospital Compare database, we calculated hospital ranks using several versions of DBWs and 2 BLVMs. Estimates in 1 BLVM were driven by differences in variances of the QIs (BLVM1) and estimates in the other by differences in the signal-to-noise ratios of the QIs (BLVM2). RESULTS: There was a high correlation in ranks among all of the DBW approaches and between those approaches and BLVM1. However, a high correlation does not necessarily mean that the same hospitals were ranked in the top or bottom quality deciles. In general, large hospitals were ranked in higher quality deciles by all of the approaches, though the effect was most apparent using BLVM2. CONCLUSIONS: Both conceptually and practically, hospital-specific DBWs are a reasonable approach for calculating a composite measure. However, this approach fails to take into account differences in the reliability of estimates from hospitals of different sizes, a big advantage of the Bayesian models.

KW - Bayesian inference

KW - Quality measurement

KW - Quality performance

UR - http://www.scopus.com/inward/record.url?scp=49649109681&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=49649109681&partnerID=8YFLogxK

U2 - 10.1097/MLR.0b013e31817893dc

DO - 10.1097/MLR.0b013e31817893dc

M3 - Article

C2 - 18665057

AN - SCOPUS:49649109681

VL - 46

SP - 778

EP - 785

JO - Medical Care

JF - Medical Care

SN - 0025-7079

IS - 8

ER -