Rating of Included Trials on the Efficacy-Effectiveness Spectrum: Development of a new tool for systematic reviews

L. Susan Wieland; Brian M. Berman; Douglas G. Altman; Jürgen Barth; Lex M. Bouter; Christopher R. D'Adamo; Klaus Linde; David Moher; C. Daniel Mullins; Shaun Treweek; Sean Tunis; Danielle A. van der Windt; Merrick Zwarenstein; Claudia Witt

doi:10.1016/j.jclinepi.2017.01.010

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: Development of a new tool for systematic reviews

L. Susan Wieland, Brian M. Berman, Douglas G. Altman, Jürgen Barth, Lex M. Bouter, Christopher R. D'Adamo, Klaus Linde, David Moher, C. Daniel Mullins, Shaun Treweek, Sean Tunis, Danielle A. van der Windt, Merrick Zwarenstein, Claudia Witt

School of Medicine

Research output: Contribution to journal › Article › peer-review

10 Scopus citations

Abstract

Background and Objective: Randomized trials may be designed to provide evidence more strongly related to efficacy or effectiveness of an intervention. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of the included trials. The objective of this study was to develop a tool to characterize randomized trials included in a systematic review on an efficacy-effectiveness continuum. Methods: We extracted rating domains and descriptors from existing tools and used a modified Delphi procedure to condense the domains and develop a new tool. The feasibility and interrater reliability of the tool was tested on trials from four systematic reviews. Results: The Rating of Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool rates clinical trials on a five-point Likert scale in four domains: (1) participant characteristics, (2) trial setting, (3) flexibility of interventions, and (4) clinical relevance of interventions. When RITES was piloted on trials from three reviews by unaffiliated raters, ratings were variable (intraclass correlation coefficient [ICC] 0.25-0.66 for the four domains); but, when RITES was used on one review by the review authors with expertise on the topic, the ratings were consistent (ICCs > 0.80. Conclusion: RITES may help to characterize the efficacy-effectiveness nature of trials included in systematic reviews.

Original language	English (US)
Journal	Journal of Clinical Epidemiology
DOIs	https://doi.org/10.1016/j.jclinepi.2017.01.010
State	Accepted/In press - 2017

Keywords

Applicability
Comparative effectiveness research
Effectiveness
Efficacy
Explanatory trial
Pragmatic trial
Randomized controlled trials
Systematic reviews

ASJC Scopus subject areas

Epidemiology

Access to Document

10.1016/j.jclinepi.2017.01.010

Cite this

Wieland, L. S., Berman, B. M., Altman, D. G., Barth, J., Bouter, L. M., D'Adamo, C. R., Linde, K., Moher, D., Mullins, C. D., Treweek, S., Tunis, S., van der Windt, D. A., Zwarenstein, M., & Witt, C. (Accepted/In press). Rating of Included Trials on the Efficacy-Effectiveness Spectrum: Development of a new tool for systematic reviews. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2017.01.010

Wieland, LS, Berman, BM, Altman, DG, Barth, J, Bouter, LM, D'Adamo, CR, Linde, K, Moher, D, Mullins, CD, Treweek, S, Tunis, S, van der Windt, DA, Zwarenstein, M & Witt, C 2017, 'Rating of Included Trials on the Efficacy-Effectiveness Spectrum: Development of a new tool for systematic reviews', Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2017.01.010

@article{64b6e692b7e04b8baea9722d33f461f2,

title = "Rating of Included Trials on the Efficacy-Effectiveness Spectrum: Development of a new tool for systematic reviews",

abstract = "Background and Objective: Randomized trials may be designed to provide evidence more strongly related to efficacy or effectiveness of an intervention. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of the included trials. The objective of this study was to develop a tool to characterize randomized trials included in a systematic review on an efficacy-effectiveness continuum. Methods: We extracted rating domains and descriptors from existing tools and used a modified Delphi procedure to condense the domains and develop a new tool. The feasibility and interrater reliability of the tool was tested on trials from four systematic reviews. Results: The Rating of Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool rates clinical trials on a five-point Likert scale in four domains: (1) participant characteristics, (2) trial setting, (3) flexibility of interventions, and (4) clinical relevance of interventions. When RITES was piloted on trials from three reviews by unaffiliated raters, ratings were variable (intraclass correlation coefficient [ICC] 0.25-0.66 for the four domains); but, when RITES was used on one review by the review authors with expertise on the topic, the ratings were consistent (ICCs > 0.80. Conclusion: RITES may help to characterize the efficacy-effectiveness nature of trials included in systematic reviews.",

keywords = "Applicability, Comparative effectiveness research, Effectiveness, Efficacy, Explanatory trial, Pragmatic trial, Randomized controlled trials, Systematic reviews",

author = "Wieland, {L. Susan} and Berman, {Brian M.} and Altman, {Douglas G.} and J{\"u}rgen Barth and Bouter, {Lex M.} and D'Adamo, {Christopher R.} and Klaus Linde and David Moher and Mullins, {C. Daniel} and Shaun Treweek and Sean Tunis and {van der Windt}, {Danielle A.} and Merrick Zwarenstein and Claudia Witt",

year = "2017",

doi = "10.1016/j.jclinepi.2017.01.010",

language = "English (US)",

journal = "Journal of Chronic Diseases",

issn = "0895-4356",

publisher = "Elsevier USA",

}

TY - JOUR

T1 - Rating of Included Trials on the Efficacy-Effectiveness Spectrum

T2 - Development of a new tool for systematic reviews

AU - Wieland, L. Susan

AU - Berman, Brian M.

AU - Altman, Douglas G.

AU - Barth, Jürgen

AU - Bouter, Lex M.

AU - D'Adamo, Christopher R.

AU - Linde, Klaus

AU - Moher, David

AU - Mullins, C. Daniel

AU - Treweek, Shaun

AU - Tunis, Sean

AU - van der Windt, Danielle A.

AU - Zwarenstein, Merrick

AU - Witt, Claudia

PY - 2017

Y1 - 2017

N2 - Background and Objective: Randomized trials may be designed to provide evidence more strongly related to efficacy or effectiveness of an intervention. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of the included trials. The objective of this study was to develop a tool to characterize randomized trials included in a systematic review on an efficacy-effectiveness continuum. Methods: We extracted rating domains and descriptors from existing tools and used a modified Delphi procedure to condense the domains and develop a new tool. The feasibility and interrater reliability of the tool was tested on trials from four systematic reviews. Results: The Rating of Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool rates clinical trials on a five-point Likert scale in four domains: (1) participant characteristics, (2) trial setting, (3) flexibility of interventions, and (4) clinical relevance of interventions. When RITES was piloted on trials from three reviews by unaffiliated raters, ratings were variable (intraclass correlation coefficient [ICC] 0.25-0.66 for the four domains); but, when RITES was used on one review by the review authors with expertise on the topic, the ratings were consistent (ICCs > 0.80. Conclusion: RITES may help to characterize the efficacy-effectiveness nature of trials included in systematic reviews.

AB - Background and Objective: Randomized trials may be designed to provide evidence more strongly related to efficacy or effectiveness of an intervention. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of the included trials. The objective of this study was to develop a tool to characterize randomized trials included in a systematic review on an efficacy-effectiveness continuum. Methods: We extracted rating domains and descriptors from existing tools and used a modified Delphi procedure to condense the domains and develop a new tool. The feasibility and interrater reliability of the tool was tested on trials from four systematic reviews. Results: The Rating of Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool rates clinical trials on a five-point Likert scale in four domains: (1) participant characteristics, (2) trial setting, (3) flexibility of interventions, and (4) clinical relevance of interventions. When RITES was piloted on trials from three reviews by unaffiliated raters, ratings were variable (intraclass correlation coefficient [ICC] 0.25-0.66 for the four domains); but, when RITES was used on one review by the review authors with expertise on the topic, the ratings were consistent (ICCs > 0.80. Conclusion: RITES may help to characterize the efficacy-effectiveness nature of trials included in systematic reviews.

KW - Applicability

KW - Comparative effectiveness research

KW - Effectiveness

KW - Efficacy

KW - Explanatory trial

KW - Pragmatic trial

KW - Randomized controlled trials

KW - Systematic reviews

UR - http://www.scopus.com/inward/record.url?scp=85016498254&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85016498254&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2017.01.010

DO - 10.1016/j.jclinepi.2017.01.010

M3 - Article

C2 - 28188898

AN - SCOPUS:85016498254

SN - 0895-4356

JO - Journal of Chronic Diseases

JF - Journal of Chronic Diseases

ER -

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: Development of a new tool for systematic reviews

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this