It's all about balance: propensity score matching in the context of complex survey data

David Lenis, Trang Nguyen, Nianbo Dong, Elizabeth Stuart

Research output: Contribution to journalArticle

Abstract

Many research studies aim to draw causal inferences using data from large, nationally representative survey samples, and many of these studies use propensity score matching to make those causal inferences as rigorous as possible given the non-experimental nature of the data. However, very few applied studies are careful about incorporating the survey design with the propensity score analysis, which may mean that the results do not generate population inferences. This may be because few methodological studies examine how to best combine these methods. Furthermore, even fewer of them investigate different non-response mechanisms. This study examines methods for handling survey weights in propensity score matching analyses of survey data under different non-response mechanisms. Our main conclusions are: (i) whether the survey weights are incorporated in the estimation of the propensity score does not impact estimation of the population treatment effect, as long as good population treated-comparison balance is achieved on confounders, (ii) survey weights must be used in the outcome analysis, and (iii) the transferring of survey weights (i.e., assigning the weights of the treated units to the comparison units matched to them) can be beneficial under certain non-response mechanisms.

Original languageEnglish (US)
Pages (from-to)147-163
Number of pages17
JournalBiostatistics (Oxford, England)
Volume20
Issue number1
DOIs
StatePublished - Jan 1 2019

Fingerprint

Propensity Score
Survey Data
Non-response
Causal Inference
Survey Design
Sample Survey
Unit
Treatment Effects
Context
Propensity score matching
Survey data

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

It's all about balance : propensity score matching in the context of complex survey data. / Lenis, David; Nguyen, Trang; Dong, Nianbo; Stuart, Elizabeth.

In: Biostatistics (Oxford, England), Vol. 20, No. 1, 01.01.2019, p. 147-163.

Research output: Contribution to journalArticle

@article{564c2d484ae24cb0a6b4d0abbf356c4c,
title = "It's all about balance: propensity score matching in the context of complex survey data",
abstract = "Many research studies aim to draw causal inferences using data from large, nationally representative survey samples, and many of these studies use propensity score matching to make those causal inferences as rigorous as possible given the non-experimental nature of the data. However, very few applied studies are careful about incorporating the survey design with the propensity score analysis, which may mean that the results do not generate population inferences. This may be because few methodological studies examine how to best combine these methods. Furthermore, even fewer of them investigate different non-response mechanisms. This study examines methods for handling survey weights in propensity score matching analyses of survey data under different non-response mechanisms. Our main conclusions are: (i) whether the survey weights are incorporated in the estimation of the propensity score does not impact estimation of the population treatment effect, as long as good population treated-comparison balance is achieved on confounders, (ii) survey weights must be used in the outcome analysis, and (iii) the transferring of survey weights (i.e., assigning the weights of the treated units to the comparison units matched to them) can be beneficial under certain non-response mechanisms.",
author = "David Lenis and Trang Nguyen and Nianbo Dong and Elizabeth Stuart",
year = "2019",
month = "1",
day = "1",
doi = "10.1093/biostatistics/kxx063",
language = "English (US)",
volume = "20",
pages = "147--163",
journal = "Biostatistics",
issn = "1465-4644",
publisher = "Oxford University Press",
number = "1",

}

TY - JOUR

T1 - It's all about balance

T2 - propensity score matching in the context of complex survey data

AU - Lenis, David

AU - Nguyen, Trang

AU - Dong, Nianbo

AU - Stuart, Elizabeth

PY - 2019/1/1

Y1 - 2019/1/1

N2 - Many research studies aim to draw causal inferences using data from large, nationally representative survey samples, and many of these studies use propensity score matching to make those causal inferences as rigorous as possible given the non-experimental nature of the data. However, very few applied studies are careful about incorporating the survey design with the propensity score analysis, which may mean that the results do not generate population inferences. This may be because few methodological studies examine how to best combine these methods. Furthermore, even fewer of them investigate different non-response mechanisms. This study examines methods for handling survey weights in propensity score matching analyses of survey data under different non-response mechanisms. Our main conclusions are: (i) whether the survey weights are incorporated in the estimation of the propensity score does not impact estimation of the population treatment effect, as long as good population treated-comparison balance is achieved on confounders, (ii) survey weights must be used in the outcome analysis, and (iii) the transferring of survey weights (i.e., assigning the weights of the treated units to the comparison units matched to them) can be beneficial under certain non-response mechanisms.

AB - Many research studies aim to draw causal inferences using data from large, nationally representative survey samples, and many of these studies use propensity score matching to make those causal inferences as rigorous as possible given the non-experimental nature of the data. However, very few applied studies are careful about incorporating the survey design with the propensity score analysis, which may mean that the results do not generate population inferences. This may be because few methodological studies examine how to best combine these methods. Furthermore, even fewer of them investigate different non-response mechanisms. This study examines methods for handling survey weights in propensity score matching analyses of survey data under different non-response mechanisms. Our main conclusions are: (i) whether the survey weights are incorporated in the estimation of the propensity score does not impact estimation of the population treatment effect, as long as good population treated-comparison balance is achieved on confounders, (ii) survey weights must be used in the outcome analysis, and (iii) the transferring of survey weights (i.e., assigning the weights of the treated units to the comparison units matched to them) can be beneficial under certain non-response mechanisms.

UR - http://www.scopus.com/inward/record.url?scp=85058725008&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058725008&partnerID=8YFLogxK

U2 - 10.1093/biostatistics/kxx063

DO - 10.1093/biostatistics/kxx063

M3 - Article

C2 - 29293896

AN - SCOPUS:85058725008

VL - 20

SP - 147

EP - 163

JO - Biostatistics

JF - Biostatistics

SN - 1465-4644

IS - 1

ER -