Prominent medical journals often provide insufficient information to assess the validity of studies with negative results

Randy S. Hebert, Scott Wright, Robert S. Dittus, Tom A. Elasy

Research output: Contribution to journalArticle

Abstract

Background: Physicians reading the medical literature attempt to determine whether research studies are valid. However, articles with negative results may not provide sufficient information to allow physicians to properly assess validity. Methods: We analyzed all original research articles with negative results published in 1997 in the weekly journals BMJ, JAMA, Lancet, and New England Journal of Medicine as well as those published in the 1997 and 1998 issues of the bimonthly Annals of Internal Medicine (N = 234). Our primary objective was to quantify the proportion of studies with negative results that comment on power and present confidence intervals. Secondary outcomes were to quantify the proportion of these studies with a specified effect size and a defined primary outcome. Stratified analyses by study design were also performed. Results: Only 30% of the articles with negative results comment on power. The reporting of power (range: 15%-52%) and confidence intervals (range: 55-81%) varied significantly among journals. Observational studies of etiology/risk factors addressed power less frequently (15%, 95% CI, 8-21%) than did clinical trials (56%, 95% CI, 46-67%, p <0.001). While 87% of articles with power calculations specified an effect size the authors sought to detect, a minority gave a rationale for the effect size. Only half of the studies with negative results clearly defined a primary outcome. Conclusion: Prominent medical journals often provide insufficient information to assess the validity of studies with negative results.

Original languageEnglish (US)
Article number1
Pages (from-to)1-5
Number of pages5
JournalJournal of Negative Results in BioMedicine
Volume1
DOIs
StatePublished - Sep 30 2002

Fingerprint

Confidence Intervals
Medicine
Physicians
New England
Internal Medicine
Research
Observational Studies
Reading
Clinical Trials

ASJC Scopus subject areas

  • Medicine(all)
  • Pharmacology, Toxicology and Pharmaceutics(all)

Cite this

Prominent medical journals often provide insufficient information to assess the validity of studies with negative results. / Hebert, Randy S.; Wright, Scott; Dittus, Robert S.; Elasy, Tom A.

In: Journal of Negative Results in BioMedicine, Vol. 1, 1, 30.09.2002, p. 1-5.

Research output: Contribution to journalArticle

@article{de2590e62daf4b3597c697acb6c6e040,
title = "Prominent medical journals often provide insufficient information to assess the validity of studies with negative results",
abstract = "Background: Physicians reading the medical literature attempt to determine whether research studies are valid. However, articles with negative results may not provide sufficient information to allow physicians to properly assess validity. Methods: We analyzed all original research articles with negative results published in 1997 in the weekly journals BMJ, JAMA, Lancet, and New England Journal of Medicine as well as those published in the 1997 and 1998 issues of the bimonthly Annals of Internal Medicine (N = 234). Our primary objective was to quantify the proportion of studies with negative results that comment on power and present confidence intervals. Secondary outcomes were to quantify the proportion of these studies with a specified effect size and a defined primary outcome. Stratified analyses by study design were also performed. Results: Only 30{\%} of the articles with negative results comment on power. The reporting of power (range: 15{\%}-52{\%}) and confidence intervals (range: 55-81{\%}) varied significantly among journals. Observational studies of etiology/risk factors addressed power less frequently (15{\%}, 95{\%} CI, 8-21{\%}) than did clinical trials (56{\%}, 95{\%} CI, 46-67{\%}, p <0.001). While 87{\%} of articles with power calculations specified an effect size the authors sought to detect, a minority gave a rationale for the effect size. Only half of the studies with negative results clearly defined a primary outcome. Conclusion: Prominent medical journals often provide insufficient information to assess the validity of studies with negative results.",
author = "Hebert, {Randy S.} and Scott Wright and Dittus, {Robert S.} and Elasy, {Tom A.}",
year = "2002",
month = "9",
day = "30",
doi = "10.1186/1477-5751-1-1",
language = "English (US)",
volume = "1",
pages = "1--5",
journal = "Journal of Negative Results in BioMedicine",
issn = "1477-5751",
publisher = "BioMed Central",

}

TY - JOUR

T1 - Prominent medical journals often provide insufficient information to assess the validity of studies with negative results

AU - Hebert, Randy S.

AU - Wright, Scott

AU - Dittus, Robert S.

AU - Elasy, Tom A.

PY - 2002/9/30

Y1 - 2002/9/30

N2 - Background: Physicians reading the medical literature attempt to determine whether research studies are valid. However, articles with negative results may not provide sufficient information to allow physicians to properly assess validity. Methods: We analyzed all original research articles with negative results published in 1997 in the weekly journals BMJ, JAMA, Lancet, and New England Journal of Medicine as well as those published in the 1997 and 1998 issues of the bimonthly Annals of Internal Medicine (N = 234). Our primary objective was to quantify the proportion of studies with negative results that comment on power and present confidence intervals. Secondary outcomes were to quantify the proportion of these studies with a specified effect size and a defined primary outcome. Stratified analyses by study design were also performed. Results: Only 30% of the articles with negative results comment on power. The reporting of power (range: 15%-52%) and confidence intervals (range: 55-81%) varied significantly among journals. Observational studies of etiology/risk factors addressed power less frequently (15%, 95% CI, 8-21%) than did clinical trials (56%, 95% CI, 46-67%, p <0.001). While 87% of articles with power calculations specified an effect size the authors sought to detect, a minority gave a rationale for the effect size. Only half of the studies with negative results clearly defined a primary outcome. Conclusion: Prominent medical journals often provide insufficient information to assess the validity of studies with negative results.

AB - Background: Physicians reading the medical literature attempt to determine whether research studies are valid. However, articles with negative results may not provide sufficient information to allow physicians to properly assess validity. Methods: We analyzed all original research articles with negative results published in 1997 in the weekly journals BMJ, JAMA, Lancet, and New England Journal of Medicine as well as those published in the 1997 and 1998 issues of the bimonthly Annals of Internal Medicine (N = 234). Our primary objective was to quantify the proportion of studies with negative results that comment on power and present confidence intervals. Secondary outcomes were to quantify the proportion of these studies with a specified effect size and a defined primary outcome. Stratified analyses by study design were also performed. Results: Only 30% of the articles with negative results comment on power. The reporting of power (range: 15%-52%) and confidence intervals (range: 55-81%) varied significantly among journals. Observational studies of etiology/risk factors addressed power less frequently (15%, 95% CI, 8-21%) than did clinical trials (56%, 95% CI, 46-67%, p <0.001). While 87% of articles with power calculations specified an effect size the authors sought to detect, a minority gave a rationale for the effect size. Only half of the studies with negative results clearly defined a primary outcome. Conclusion: Prominent medical journals often provide insufficient information to assess the validity of studies with negative results.

UR - http://www.scopus.com/inward/record.url?scp=0345635453&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0345635453&partnerID=8YFLogxK

U2 - 10.1186/1477-5751-1-1

DO - 10.1186/1477-5751-1-1

M3 - Article

VL - 1

SP - 1

EP - 5

JO - Journal of Negative Results in BioMedicine

JF - Journal of Negative Results in BioMedicine

SN - 1477-5751

M1 - 1

ER -