Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis

Evan R Mayo-Wilson, Nicole Fusco, Tianjing Li, Kay Dickersin, Hwanhee Hong, Joseph K. Canner

Research output: Contribution to journalReview article

Abstract

Objective To identify variations in outcomes and results across reports of randomized clinical trials (RCTs). Study Design and Setting Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic (e.g., clinical study reports) sources by 2015. We prespecified outcome domains. From each source, we collected “outcomes” (i.e., domain, measure, metric, method of aggregation, and time point); “treatment effect” (i.e., outcome plus the methods of analysis [e.g., how missing data were handled]); and results (i.e., numerical contrasts of treatment and comparison groups). We assessed whether results included sufficient information for meta-analysis. Results We found 21 gabapentin (68 public, 6 nonpublic reports) and seven quetiapine RCTs (46 public, 4 nonpublic reports). For four (gabapentin) and seven (quetiapine) prespecified outcome domains, RCTs reported 214 and 81 outcomes by varying four elements. RCTs assessed 605 and 188 treatment effects by varying the analysis of those outcomes. RCTs reported 1,230 and 661 meta-analyzable results, 305 (25%) and 109 (16%) in public reports. Conclusion RCTs included hundreds of outcomes and results; a small proportion were in public reports. Trialists and meta-analysts may cherry-pick what they report from multiple sources of RCT information.

Original languageEnglish (US)
Pages (from-to)39-50
Number of pages12
JournalJournal of Clinical Epidemiology
Volume86
DOIs
StatePublished - Jun 1 2017

Fingerprint

Randomized Controlled Trials
Clinical Trials
Research
Neuralgia
Bipolar Disorder
Meta-Analysis
Therapeutics
gabapentin
Quetiapine Fumarate

Keywords

  • Clinical trials
  • Meta-analysis
  • Outcomes
  • Selective outcome reporting
  • Systematic reviews

ASJC Scopus subject areas

  • Epidemiology

Cite this

Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. / Mayo-Wilson, Evan R; Fusco, Nicole; Li, Tianjing; Dickersin, Kay; Hong, Hwanhee; Canner, Joseph K.

In: Journal of Clinical Epidemiology, Vol. 86, 01.06.2017, p. 39-50.

Research output: Contribution to journalReview article

@article{736187d496b84c118e3812c3d74cd8ea,
title = "Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis",
abstract = "Objective To identify variations in outcomes and results across reports of randomized clinical trials (RCTs). Study Design and Setting Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic (e.g., clinical study reports) sources by 2015. We prespecified outcome domains. From each source, we collected “outcomes” (i.e., domain, measure, metric, method of aggregation, and time point); “treatment effect” (i.e., outcome plus the methods of analysis [e.g., how missing data were handled]); and results (i.e., numerical contrasts of treatment and comparison groups). We assessed whether results included sufficient information for meta-analysis. Results We found 21 gabapentin (68 public, 6 nonpublic reports) and seven quetiapine RCTs (46 public, 4 nonpublic reports). For four (gabapentin) and seven (quetiapine) prespecified outcome domains, RCTs reported 214 and 81 outcomes by varying four elements. RCTs assessed 605 and 188 treatment effects by varying the analysis of those outcomes. RCTs reported 1,230 and 661 meta-analyzable results, 305 (25{\%}) and 109 (16{\%}) in public reports. Conclusion RCTs included hundreds of outcomes and results; a small proportion were in public reports. Trialists and meta-analysts may cherry-pick what they report from multiple sources of RCT information.",
keywords = "Clinical trials, Meta-analysis, Outcomes, Selective outcome reporting, Systematic reviews",
author = "Mayo-Wilson, {Evan R} and Nicole Fusco and Tianjing Li and Kay Dickersin and Hwanhee Hong and Canner, {Joseph K.}",
year = "2017",
month = "6",
day = "1",
doi = "10.1016/j.jclinepi.2017.05.007",
language = "English (US)",
volume = "86",
pages = "39--50",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",

}

TY - JOUR

T1 - Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis

AU - Mayo-Wilson, Evan R

AU - Fusco, Nicole

AU - Li, Tianjing

AU - Dickersin, Kay

AU - Hong, Hwanhee

AU - Canner, Joseph K.

PY - 2017/6/1

Y1 - 2017/6/1

N2 - Objective To identify variations in outcomes and results across reports of randomized clinical trials (RCTs). Study Design and Setting Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic (e.g., clinical study reports) sources by 2015. We prespecified outcome domains. From each source, we collected “outcomes” (i.e., domain, measure, metric, method of aggregation, and time point); “treatment effect” (i.e., outcome plus the methods of analysis [e.g., how missing data were handled]); and results (i.e., numerical contrasts of treatment and comparison groups). We assessed whether results included sufficient information for meta-analysis. Results We found 21 gabapentin (68 public, 6 nonpublic reports) and seven quetiapine RCTs (46 public, 4 nonpublic reports). For four (gabapentin) and seven (quetiapine) prespecified outcome domains, RCTs reported 214 and 81 outcomes by varying four elements. RCTs assessed 605 and 188 treatment effects by varying the analysis of those outcomes. RCTs reported 1,230 and 661 meta-analyzable results, 305 (25%) and 109 (16%) in public reports. Conclusion RCTs included hundreds of outcomes and results; a small proportion were in public reports. Trialists and meta-analysts may cherry-pick what they report from multiple sources of RCT information.

AB - Objective To identify variations in outcomes and results across reports of randomized clinical trials (RCTs). Study Design and Setting Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and nonpublic (e.g., clinical study reports) sources by 2015. We prespecified outcome domains. From each source, we collected “outcomes” (i.e., domain, measure, metric, method of aggregation, and time point); “treatment effect” (i.e., outcome plus the methods of analysis [e.g., how missing data were handled]); and results (i.e., numerical contrasts of treatment and comparison groups). We assessed whether results included sufficient information for meta-analysis. Results We found 21 gabapentin (68 public, 6 nonpublic reports) and seven quetiapine RCTs (46 public, 4 nonpublic reports). For four (gabapentin) and seven (quetiapine) prespecified outcome domains, RCTs reported 214 and 81 outcomes by varying four elements. RCTs assessed 605 and 188 treatment effects by varying the analysis of those outcomes. RCTs reported 1,230 and 661 meta-analyzable results, 305 (25%) and 109 (16%) in public reports. Conclusion RCTs included hundreds of outcomes and results; a small proportion were in public reports. Trialists and meta-analysts may cherry-pick what they report from multiple sources of RCT information.

KW - Clinical trials

KW - Meta-analysis

KW - Outcomes

KW - Selective outcome reporting

KW - Systematic reviews

UR - http://www.scopus.com/inward/record.url?scp=85020419684&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020419684&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2017.05.007

DO - 10.1016/j.jclinepi.2017.05.007

M3 - Review article

VL - 86

SP - 39

EP - 50

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -