Methods for evaluating natural experiments in obesity: A systematic review

Wendy Bennett, Renee Wilson, Allen Zhang, Eva Tseng, Emily Knapp, Hadi H K Kharrazi, Elizabeth Stuart, Oluwaseun Shogbesan, Eric B Bass, Lawrence J Cheskin

Research output: Contribution to journalReview article

Abstract

Background: Given the obesity pandemic, rigorous methodological approaches, including natural experiments, are needed. Purpose: To identify studies that report effects of programs, policies, or built environment changes on obesity prevention and control and to describe their methods. Data Sources: PubMed, CINAHL, PsycINFO, and EconLit (January 2000 to August 2017). Study Selection: Natural experiments and experimental studies evaluating a program, policy, or built environment change in U.S. or non-U.S. populations by using measures of obesity or obesity-related health behaviors. Data Extraction: 2 reviewers serially extracted data on study design, population characteristics, data sources and linkages, measures, and analytic methods and independently evaluated risk of bias. Data Synthesis: 294 studies (188 U.S., 106 non-U.S.) were identified, including 156 natural experiments (53%), 118 experimental studies (40%), and 20 (7%) with unclear study design. Studies used 106 (71 U.S., 35 non-U.S.) data systems; 37% of the U.S. data systems were linked to another data source. For outcomes, 112 studies reported childhood weight and 32 adult weight; 152 had physical activity and 148 had dietary measures. For analysis, natural experiments most commonly used cross-sectional comparisons of exposed and unexposed groups (n = 55 [35%]). Most natural experiments had a high risk of bias, and 63% had weak handling of withdrawals and dropouts. Limitation: Outcomes restricted to obesity measures and health behaviors; inconsistent or unclear descriptions of natural experiment designs; and imperfect methods for assessing risk of bias in natural experiments. Conclusion: Many methodologically diverse natural experiments and experimental studies were identified that reported effects of U.S. and non-U.S. programs, policies, or built environment changes on obesity prevention and control. The findings reinforce the need for methodological and analytic advances that would strengthen evaluations of obesity prevention and control initiatives.

Original languageEnglish (US)
Pages (from-to)791-800
Number of pages10
JournalAnnals of Internal Medicine
Volume168
Issue number11
DOIs
StatePublished - Jun 5 2018

Fingerprint

Obesity
Information Storage and Retrieval
Health Behavior
Information Systems
Weights and Measures
Genetic Selection
Pandemics
Population Characteristics
PubMed
Outcome Assessment (Health Care)
Population

ASJC Scopus subject areas

  • Internal Medicine

Cite this

Methods for evaluating natural experiments in obesity : A systematic review. / Bennett, Wendy; Wilson, Renee; Zhang, Allen; Tseng, Eva; Knapp, Emily; Kharrazi, Hadi H K; Stuart, Elizabeth; Shogbesan, Oluwaseun; Bass, Eric B; Cheskin, Lawrence J.

In: Annals of Internal Medicine, Vol. 168, No. 11, 05.06.2018, p. 791-800.

Research output: Contribution to journalReview article

@article{e2a362195b8243c1ab31e76ef8231152,
title = "Methods for evaluating natural experiments in obesity: A systematic review",
abstract = "Background: Given the obesity pandemic, rigorous methodological approaches, including natural experiments, are needed. Purpose: To identify studies that report effects of programs, policies, or built environment changes on obesity prevention and control and to describe their methods. Data Sources: PubMed, CINAHL, PsycINFO, and EconLit (January 2000 to August 2017). Study Selection: Natural experiments and experimental studies evaluating a program, policy, or built environment change in U.S. or non-U.S. populations by using measures of obesity or obesity-related health behaviors. Data Extraction: 2 reviewers serially extracted data on study design, population characteristics, data sources and linkages, measures, and analytic methods and independently evaluated risk of bias. Data Synthesis: 294 studies (188 U.S., 106 non-U.S.) were identified, including 156 natural experiments (53{\%}), 118 experimental studies (40{\%}), and 20 (7{\%}) with unclear study design. Studies used 106 (71 U.S., 35 non-U.S.) data systems; 37{\%} of the U.S. data systems were linked to another data source. For outcomes, 112 studies reported childhood weight and 32 adult weight; 152 had physical activity and 148 had dietary measures. For analysis, natural experiments most commonly used cross-sectional comparisons of exposed and unexposed groups (n = 55 [35{\%}]). Most natural experiments had a high risk of bias, and 63{\%} had weak handling of withdrawals and dropouts. Limitation: Outcomes restricted to obesity measures and health behaviors; inconsistent or unclear descriptions of natural experiment designs; and imperfect methods for assessing risk of bias in natural experiments. Conclusion: Many methodologically diverse natural experiments and experimental studies were identified that reported effects of U.S. and non-U.S. programs, policies, or built environment changes on obesity prevention and control. The findings reinforce the need for methodological and analytic advances that would strengthen evaluations of obesity prevention and control initiatives.",
author = "Wendy Bennett and Renee Wilson and Allen Zhang and Eva Tseng and Emily Knapp and Kharrazi, {Hadi H K} and Elizabeth Stuart and Oluwaseun Shogbesan and Bass, {Eric B} and Cheskin, {Lawrence J}",
year = "2018",
month = "6",
day = "5",
doi = "10.7326/M18-0309",
language = "English (US)",
volume = "168",
pages = "791--800",
journal = "Annals of Internal Medicine",
issn = "0003-4819",
publisher = "American College of Physicians",
number = "11",

}

TY - JOUR

T1 - Methods for evaluating natural experiments in obesity

T2 - A systematic review

AU - Bennett, Wendy

AU - Wilson, Renee

AU - Zhang, Allen

AU - Tseng, Eva

AU - Knapp, Emily

AU - Kharrazi, Hadi H K

AU - Stuart, Elizabeth

AU - Shogbesan, Oluwaseun

AU - Bass, Eric B

AU - Cheskin, Lawrence J

PY - 2018/6/5

Y1 - 2018/6/5

N2 - Background: Given the obesity pandemic, rigorous methodological approaches, including natural experiments, are needed. Purpose: To identify studies that report effects of programs, policies, or built environment changes on obesity prevention and control and to describe their methods. Data Sources: PubMed, CINAHL, PsycINFO, and EconLit (January 2000 to August 2017). Study Selection: Natural experiments and experimental studies evaluating a program, policy, or built environment change in U.S. or non-U.S. populations by using measures of obesity or obesity-related health behaviors. Data Extraction: 2 reviewers serially extracted data on study design, population characteristics, data sources and linkages, measures, and analytic methods and independently evaluated risk of bias. Data Synthesis: 294 studies (188 U.S., 106 non-U.S.) were identified, including 156 natural experiments (53%), 118 experimental studies (40%), and 20 (7%) with unclear study design. Studies used 106 (71 U.S., 35 non-U.S.) data systems; 37% of the U.S. data systems were linked to another data source. For outcomes, 112 studies reported childhood weight and 32 adult weight; 152 had physical activity and 148 had dietary measures. For analysis, natural experiments most commonly used cross-sectional comparisons of exposed and unexposed groups (n = 55 [35%]). Most natural experiments had a high risk of bias, and 63% had weak handling of withdrawals and dropouts. Limitation: Outcomes restricted to obesity measures and health behaviors; inconsistent or unclear descriptions of natural experiment designs; and imperfect methods for assessing risk of bias in natural experiments. Conclusion: Many methodologically diverse natural experiments and experimental studies were identified that reported effects of U.S. and non-U.S. programs, policies, or built environment changes on obesity prevention and control. The findings reinforce the need for methodological and analytic advances that would strengthen evaluations of obesity prevention and control initiatives.

AB - Background: Given the obesity pandemic, rigorous methodological approaches, including natural experiments, are needed. Purpose: To identify studies that report effects of programs, policies, or built environment changes on obesity prevention and control and to describe their methods. Data Sources: PubMed, CINAHL, PsycINFO, and EconLit (January 2000 to August 2017). Study Selection: Natural experiments and experimental studies evaluating a program, policy, or built environment change in U.S. or non-U.S. populations by using measures of obesity or obesity-related health behaviors. Data Extraction: 2 reviewers serially extracted data on study design, population characteristics, data sources and linkages, measures, and analytic methods and independently evaluated risk of bias. Data Synthesis: 294 studies (188 U.S., 106 non-U.S.) were identified, including 156 natural experiments (53%), 118 experimental studies (40%), and 20 (7%) with unclear study design. Studies used 106 (71 U.S., 35 non-U.S.) data systems; 37% of the U.S. data systems were linked to another data source. For outcomes, 112 studies reported childhood weight and 32 adult weight; 152 had physical activity and 148 had dietary measures. For analysis, natural experiments most commonly used cross-sectional comparisons of exposed and unexposed groups (n = 55 [35%]). Most natural experiments had a high risk of bias, and 63% had weak handling of withdrawals and dropouts. Limitation: Outcomes restricted to obesity measures and health behaviors; inconsistent or unclear descriptions of natural experiment designs; and imperfect methods for assessing risk of bias in natural experiments. Conclusion: Many methodologically diverse natural experiments and experimental studies were identified that reported effects of U.S. and non-U.S. programs, policies, or built environment changes on obesity prevention and control. The findings reinforce the need for methodological and analytic advances that would strengthen evaluations of obesity prevention and control initiatives.

UR - http://www.scopus.com/inward/record.url?scp=85048497145&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85048497145&partnerID=8YFLogxK

U2 - 10.7326/M18-0309

DO - 10.7326/M18-0309

M3 - Review article

C2 - 29710087

AN - SCOPUS:85048497145

VL - 168

SP - 791

EP - 800

JO - Annals of Internal Medicine

JF - Annals of Internal Medicine

SN - 0003-4819

IS - 11

ER -