Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions

Roger Chou, Rongwei Fu, Susan Carson, Somnath Saha, Mark Helfand

Research output: Contribution to journalArticle

Abstract

Objectives: High quality harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Little is known, however, about whether perceived methodological shortcomings are associated with lower estimates of harms. Study Design and Setting: Studies reporting harms associated with carotid endarterectomy (CEA) and rofecoxib were identified using published systematic reviews. A standardized abstraction form, including eight predefined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms. Results: In 111 studies of CEA, meeting five of the eight-quality criteria was associated with significantly higher adverse event rates. A quality-rating instrument with four criteria predicted adverse events (5.7% in studies rated "adequate," compared to 3.9% in studies rated "inadequate" [P = 0.0003]). In multivariate analyses, the quality-rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted estimates of risk for myocardial infarction in 16 trials of rofecoxib. Conclusion: The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should consider methodological shortcomings when evaluating estimates of harms associated with clinical interventions.

Original languageEnglish (US)
Pages (from-to)18-28
Number of pages11
JournalJournal of Clinical Epidemiology
Volume60
Issue number1
DOIs
StatePublished - Jan 2007
Externally publishedYes

Fingerprint

Carotid Endarterectomy
Multivariate Analysis
Myocardial Infarction
Research Personnel
Clinical Studies
rofecoxib
Data Accuracy

Keywords

  • Carotid
  • Cerebrovascular accident
  • Cyclooxygenase-2 inhibitors
  • Endarterectomy
  • Meta-analysis
  • Postoperative complications
  • Regression analysis

ASJC Scopus subject areas

  • Medicine(all)
  • Public Health, Environmental and Occupational Health
  • Epidemiology

Cite this

Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions. / Chou, Roger; Fu, Rongwei; Carson, Susan; Saha, Somnath; Helfand, Mark.

In: Journal of Clinical Epidemiology, Vol. 60, No. 1, 01.2007, p. 18-28.

Research output: Contribution to journalArticle

Chou, Roger ; Fu, Rongwei ; Carson, Susan ; Saha, Somnath ; Helfand, Mark. / Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions. In: Journal of Clinical Epidemiology. 2007 ; Vol. 60, No. 1. pp. 18-28.
@article{4f0ff663e04b477bbfd3069ea1d000c5,
title = "Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions",
abstract = "Objectives: High quality harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Little is known, however, about whether perceived methodological shortcomings are associated with lower estimates of harms. Study Design and Setting: Studies reporting harms associated with carotid endarterectomy (CEA) and rofecoxib were identified using published systematic reviews. A standardized abstraction form, including eight predefined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms. Results: In 111 studies of CEA, meeting five of the eight-quality criteria was associated with significantly higher adverse event rates. A quality-rating instrument with four criteria predicted adverse events (5.7{\%} in studies rated {"}adequate,{"} compared to 3.9{\%} in studies rated {"}inadequate{"} [P = 0.0003]). In multivariate analyses, the quality-rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted estimates of risk for myocardial infarction in 16 trials of rofecoxib. Conclusion: The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should consider methodological shortcomings when evaluating estimates of harms associated with clinical interventions.",
keywords = "Carotid, Cerebrovascular accident, Cyclooxygenase-2 inhibitors, Endarterectomy, Meta-analysis, Postoperative complications, Regression analysis",
author = "Roger Chou and Rongwei Fu and Susan Carson and Somnath Saha and Mark Helfand",
year = "2007",
month = "1",
doi = "10.1016/j.jclinepi.2006.02.021",
language = "English (US)",
volume = "60",
pages = "18--28",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "1",

}

TY - JOUR

T1 - Methodological shortcomings predicted lower harm estimates in one of two sets of studies of clinical interventions

AU - Chou, Roger

AU - Fu, Rongwei

AU - Carson, Susan

AU - Saha, Somnath

AU - Helfand, Mark

PY - 2007/1

Y1 - 2007/1

N2 - Objectives: High quality harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Little is known, however, about whether perceived methodological shortcomings are associated with lower estimates of harms. Study Design and Setting: Studies reporting harms associated with carotid endarterectomy (CEA) and rofecoxib were identified using published systematic reviews. A standardized abstraction form, including eight predefined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms. Results: In 111 studies of CEA, meeting five of the eight-quality criteria was associated with significantly higher adverse event rates. A quality-rating instrument with four criteria predicted adverse events (5.7% in studies rated "adequate," compared to 3.9% in studies rated "inadequate" [P = 0.0003]). In multivariate analyses, the quality-rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted estimates of risk for myocardial infarction in 16 trials of rofecoxib. Conclusion: The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should consider methodological shortcomings when evaluating estimates of harms associated with clinical interventions.

AB - Objectives: High quality harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Little is known, however, about whether perceived methodological shortcomings are associated with lower estimates of harms. Study Design and Setting: Studies reporting harms associated with carotid endarterectomy (CEA) and rofecoxib were identified using published systematic reviews. A standardized abstraction form, including eight predefined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms. Results: In 111 studies of CEA, meeting five of the eight-quality criteria was associated with significantly higher adverse event rates. A quality-rating instrument with four criteria predicted adverse events (5.7% in studies rated "adequate," compared to 3.9% in studies rated "inadequate" [P = 0.0003]). In multivariate analyses, the quality-rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted estimates of risk for myocardial infarction in 16 trials of rofecoxib. Conclusion: The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should consider methodological shortcomings when evaluating estimates of harms associated with clinical interventions.

KW - Carotid

KW - Cerebrovascular accident

KW - Cyclooxygenase-2 inhibitors

KW - Endarterectomy

KW - Meta-analysis

KW - Postoperative complications

KW - Regression analysis

UR - http://www.scopus.com/inward/record.url?scp=33845213185&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33845213185&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2006.02.021

DO - 10.1016/j.jclinepi.2006.02.021

M3 - Article

C2 - 17161750

AN - SCOPUS:33845213185

VL - 60

SP - 18

EP - 28

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

IS - 1

ER -