Generalizability of the findings from a randomized controlled trial of a web-based substance use disorder intervention

Research output: Contribution to journalArticle

Abstract

Background and Objectives: There is growing concern regarding the generalizability of findings from randomized controlled trials (RCTs) of interventions for substance use disorders (SUDs). This study used a selection model approach to assess and improve the generalizability of an evaluation for a web-based SUD intervention by making the trial sample resemble the target population. Methods: The sample of the web-based SUD intervention (Therapeutic Education System vs. Treatment-as-usual; n = 507) was compared with the target population of SUD treatment-seeking individuals from the Treatment Episodes Data Set-Admissions (TEDS-A). Using weights based on the probabilities of RCT participation, we computed weighted treatment effects on retention and abstinence. Results: Substantial differences between the RCT sample and the target population was demonstrated in significant difference in the mean propensity scores (1.62 standard deviations at p <.001). The population effect on abstinence (12 weeks and 6 months) was statistically insignificant after weighting the data with the generalizability weight. Discussions and Conclusions: Generalizability of the findings from the RCT could be limited when the RCT sample does not well represent the target population. Scientific Significance: Application of generalizability weights can be a potentially useful tool to improve generalizability of RCT findings. (Am J Addict 2018;27:231–237).

Original languageEnglish (US)
Pages (from-to)231-237
Number of pages7
JournalAmerican Journal on Addictions
Volume27
Issue number3
DOIs
StatePublished - Apr 1 2018

Fingerprint

Substance-Related Disorders
Randomized Controlled Trials
Health Services Needs and Demand
Weights and Measures
Propensity Score
Education
Population

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Clinical Psychology
  • Psychiatry and Mental health

Cite this

@article{d78f2c5ffd8e4481bd7c94ce40e1c7b5,
title = "Generalizability of the findings from a randomized controlled trial of a web-based substance use disorder intervention",
abstract = "Background and Objectives: There is growing concern regarding the generalizability of findings from randomized controlled trials (RCTs) of interventions for substance use disorders (SUDs). This study used a selection model approach to assess and improve the generalizability of an evaluation for a web-based SUD intervention by making the trial sample resemble the target population. Methods: The sample of the web-based SUD intervention (Therapeutic Education System vs. Treatment-as-usual; n = 507) was compared with the target population of SUD treatment-seeking individuals from the Treatment Episodes Data Set-Admissions (TEDS-A). Using weights based on the probabilities of RCT participation, we computed weighted treatment effects on retention and abstinence. Results: Substantial differences between the RCT sample and the target population was demonstrated in significant difference in the mean propensity scores (1.62 standard deviations at p <.001). The population effect on abstinence (12 weeks and 6 months) was statistically insignificant after weighting the data with the generalizability weight. Discussions and Conclusions: Generalizability of the findings from the RCT could be limited when the RCT sample does not well represent the target population. Scientific Significance: Application of generalizability weights can be a potentially useful tool to improve generalizability of RCT findings. (Am J Addict 2018;27:231–237).",
author = "Ryoko Susukida and Crum, {Rosa M} and Elizabeth Stuart and Ramin Mojtabai",
year = "2018",
month = "4",
day = "1",
doi = "10.1111/ajad.12714",
language = "English (US)",
volume = "27",
pages = "231--237",
journal = "American Journal on Addictions",
issn = "1055-0496",
publisher = "Wiley-Blackwell",
number = "3",

}

TY - JOUR

T1 - Generalizability of the findings from a randomized controlled trial of a web-based substance use disorder intervention

AU - Susukida, Ryoko

AU - Crum, Rosa M

AU - Stuart, Elizabeth

AU - Mojtabai, Ramin

PY - 2018/4/1

Y1 - 2018/4/1

N2 - Background and Objectives: There is growing concern regarding the generalizability of findings from randomized controlled trials (RCTs) of interventions for substance use disorders (SUDs). This study used a selection model approach to assess and improve the generalizability of an evaluation for a web-based SUD intervention by making the trial sample resemble the target population. Methods: The sample of the web-based SUD intervention (Therapeutic Education System vs. Treatment-as-usual; n = 507) was compared with the target population of SUD treatment-seeking individuals from the Treatment Episodes Data Set-Admissions (TEDS-A). Using weights based on the probabilities of RCT participation, we computed weighted treatment effects on retention and abstinence. Results: Substantial differences between the RCT sample and the target population was demonstrated in significant difference in the mean propensity scores (1.62 standard deviations at p <.001). The population effect on abstinence (12 weeks and 6 months) was statistically insignificant after weighting the data with the generalizability weight. Discussions and Conclusions: Generalizability of the findings from the RCT could be limited when the RCT sample does not well represent the target population. Scientific Significance: Application of generalizability weights can be a potentially useful tool to improve generalizability of RCT findings. (Am J Addict 2018;27:231–237).

AB - Background and Objectives: There is growing concern regarding the generalizability of findings from randomized controlled trials (RCTs) of interventions for substance use disorders (SUDs). This study used a selection model approach to assess and improve the generalizability of an evaluation for a web-based SUD intervention by making the trial sample resemble the target population. Methods: The sample of the web-based SUD intervention (Therapeutic Education System vs. Treatment-as-usual; n = 507) was compared with the target population of SUD treatment-seeking individuals from the Treatment Episodes Data Set-Admissions (TEDS-A). Using weights based on the probabilities of RCT participation, we computed weighted treatment effects on retention and abstinence. Results: Substantial differences between the RCT sample and the target population was demonstrated in significant difference in the mean propensity scores (1.62 standard deviations at p <.001). The population effect on abstinence (12 weeks and 6 months) was statistically insignificant after weighting the data with the generalizability weight. Discussions and Conclusions: Generalizability of the findings from the RCT could be limited when the RCT sample does not well represent the target population. Scientific Significance: Application of generalizability weights can be a potentially useful tool to improve generalizability of RCT findings. (Am J Addict 2018;27:231–237).

UR - http://www.scopus.com/inward/record.url?scp=85044182461&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85044182461&partnerID=8YFLogxK

U2 - 10.1111/ajad.12714

DO - 10.1111/ajad.12714

M3 - Article

C2 - 29569403

AN - SCOPUS:85044182461

VL - 27

SP - 231

EP - 237

JO - American Journal on Addictions

JF - American Journal on Addictions

SN - 1055-0496

IS - 3

ER -