Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions

Christopher W. Halladay, Thomas A. Trikalinos, Ian Schmid, Christopher H. Schmid, Issa J. Dahabreh

Research output: Contribution to journalArticle

Abstract

Objectives Searching multiple sources when conducting systematic reviews is considered good practice. We aimed to investigate the impact of using sources beyond PubMed in systematic reviews of therapeutic interventions. Study Design and Setting We randomly selected 50 Cochrane reviews that searched the PubMed (or MEDLINE) and EMBASE databases and included a meta-analysis of ≥10 studies. We checked whether each eligible record in each review (n = 2,700) was retrievable in PubMed and EMBASE. For the first-listed meta-analysis of ≥10 studies in each review, we examined whether excluding studies not found in PubMed affected results. Results A median of one record per review was indexed in EMBASE but not in PubMed; a median of four records per review was not indexed in PubMed or EMBASE. Meta-analyses included a median of 13.5 studies; a median of zero studies per meta-analysis was indexed in EMBASE but not in PubMed; a median of one study per meta-analysis was not indexed in PubMed or EMBASE. Meta-analysis using only PubMed-indexed vs. all available studies led to a different conclusion in a single case (on the basis of conventional criteria for statistical significance). In meta-regression analyses, effects in PubMed- vs. non-PubMed-indexed studies were statistically significantly different in a single data set. Conclusion For systematic reviews of the effects of therapeutic interventions, gains from searching sources beyond PubMed, and from searching EMBASE in particular are modest.

Original languageEnglish (US)
Pages (from-to)1076-1084
Number of pages9
JournalJournal of Clinical Epidemiology
Volume68
Issue number9
DOIs
StatePublished - Sep 1 2015
Externally publishedYes

Fingerprint

Information Storage and Retrieval
PubMed
Meta-Analysis
Therapeutics
Therapeutic Uses
MEDLINE
Regression Analysis
Databases

Keywords

  • Databases
  • EMBASE
  • Health technology assessment
  • Literature search
  • Meta-analysis
  • PubMed
  • Rapid review
  • Systematic review

ASJC Scopus subject areas

  • Epidemiology

Cite this

Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. / Halladay, Christopher W.; Trikalinos, Thomas A.; Schmid, Ian; Schmid, Christopher H.; Dahabreh, Issa J.

In: Journal of Clinical Epidemiology, Vol. 68, No. 9, 01.09.2015, p. 1076-1084.

Research output: Contribution to journalArticle

Halladay, Christopher W. ; Trikalinos, Thomas A. ; Schmid, Ian ; Schmid, Christopher H. ; Dahabreh, Issa J. / Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions. In: Journal of Clinical Epidemiology. 2015 ; Vol. 68, No. 9. pp. 1076-1084.
@article{b7d98fcbd6914788a0270f94032a58e2,
title = "Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions",
abstract = "Objectives Searching multiple sources when conducting systematic reviews is considered good practice. We aimed to investigate the impact of using sources beyond PubMed in systematic reviews of therapeutic interventions. Study Design and Setting We randomly selected 50 Cochrane reviews that searched the PubMed (or MEDLINE) and EMBASE databases and included a meta-analysis of ≥10 studies. We checked whether each eligible record in each review (n = 2,700) was retrievable in PubMed and EMBASE. For the first-listed meta-analysis of ≥10 studies in each review, we examined whether excluding studies not found in PubMed affected results. Results A median of one record per review was indexed in EMBASE but not in PubMed; a median of four records per review was not indexed in PubMed or EMBASE. Meta-analyses included a median of 13.5 studies; a median of zero studies per meta-analysis was indexed in EMBASE but not in PubMed; a median of one study per meta-analysis was not indexed in PubMed or EMBASE. Meta-analysis using only PubMed-indexed vs. all available studies led to a different conclusion in a single case (on the basis of conventional criteria for statistical significance). In meta-regression analyses, effects in PubMed- vs. non-PubMed-indexed studies were statistically significantly different in a single data set. Conclusion For systematic reviews of the effects of therapeutic interventions, gains from searching sources beyond PubMed, and from searching EMBASE in particular are modest.",
keywords = "Databases, EMBASE, Health technology assessment, Literature search, Meta-analysis, PubMed, Rapid review, Systematic review",
author = "Halladay, {Christopher W.} and Trikalinos, {Thomas A.} and Ian Schmid and Schmid, {Christopher H.} and Dahabreh, {Issa J.}",
year = "2015",
month = "9",
day = "1",
doi = "10.1016/j.jclinepi.2014.12.017",
language = "English (US)",
volume = "68",
pages = "1076--1084",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier USA",
number = "9",

}

TY - JOUR

T1 - Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions

AU - Halladay, Christopher W.

AU - Trikalinos, Thomas A.

AU - Schmid, Ian

AU - Schmid, Christopher H.

AU - Dahabreh, Issa J.

PY - 2015/9/1

Y1 - 2015/9/1

N2 - Objectives Searching multiple sources when conducting systematic reviews is considered good practice. We aimed to investigate the impact of using sources beyond PubMed in systematic reviews of therapeutic interventions. Study Design and Setting We randomly selected 50 Cochrane reviews that searched the PubMed (or MEDLINE) and EMBASE databases and included a meta-analysis of ≥10 studies. We checked whether each eligible record in each review (n = 2,700) was retrievable in PubMed and EMBASE. For the first-listed meta-analysis of ≥10 studies in each review, we examined whether excluding studies not found in PubMed affected results. Results A median of one record per review was indexed in EMBASE but not in PubMed; a median of four records per review was not indexed in PubMed or EMBASE. Meta-analyses included a median of 13.5 studies; a median of zero studies per meta-analysis was indexed in EMBASE but not in PubMed; a median of one study per meta-analysis was not indexed in PubMed or EMBASE. Meta-analysis using only PubMed-indexed vs. all available studies led to a different conclusion in a single case (on the basis of conventional criteria for statistical significance). In meta-regression analyses, effects in PubMed- vs. non-PubMed-indexed studies were statistically significantly different in a single data set. Conclusion For systematic reviews of the effects of therapeutic interventions, gains from searching sources beyond PubMed, and from searching EMBASE in particular are modest.

AB - Objectives Searching multiple sources when conducting systematic reviews is considered good practice. We aimed to investigate the impact of using sources beyond PubMed in systematic reviews of therapeutic interventions. Study Design and Setting We randomly selected 50 Cochrane reviews that searched the PubMed (or MEDLINE) and EMBASE databases and included a meta-analysis of ≥10 studies. We checked whether each eligible record in each review (n = 2,700) was retrievable in PubMed and EMBASE. For the first-listed meta-analysis of ≥10 studies in each review, we examined whether excluding studies not found in PubMed affected results. Results A median of one record per review was indexed in EMBASE but not in PubMed; a median of four records per review was not indexed in PubMed or EMBASE. Meta-analyses included a median of 13.5 studies; a median of zero studies per meta-analysis was indexed in EMBASE but not in PubMed; a median of one study per meta-analysis was not indexed in PubMed or EMBASE. Meta-analysis using only PubMed-indexed vs. all available studies led to a different conclusion in a single case (on the basis of conventional criteria for statistical significance). In meta-regression analyses, effects in PubMed- vs. non-PubMed-indexed studies were statistically significantly different in a single data set. Conclusion For systematic reviews of the effects of therapeutic interventions, gains from searching sources beyond PubMed, and from searching EMBASE in particular are modest.

KW - Databases

KW - EMBASE

KW - Health technology assessment

KW - Literature search

KW - Meta-analysis

KW - PubMed

KW - Rapid review

KW - Systematic review

UR - http://www.scopus.com/inward/record.url?scp=84938911177&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84938911177&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2014.12.017

DO - 10.1016/j.jclinepi.2014.12.017

M3 - Article

C2 - 26279401

AN - SCOPUS:84938911177

VL - 68

SP - 1076

EP - 1084

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

IS - 9

ER -