Optimizing matching and analysis combinations for estimating causal effects

K. Ellicott Colson, Kara E. Rudolph, Scott C. Zimmerman, Dana E. Goin, Elizabeth Stuart, Mark Van Der Laan, Jennifer Ahern

Research output: Contribution to journalArticle

Abstract

Matching methods are common in studies across many disciplines. However, there is limited evidence on how to optimally combine matching with subsequent analysis approaches to minimize bias and maximize efficiency for the quantity of interest. We conducted simulations to compare the performance of a wide variety of matching methods and analysis approaches in terms of bias, variance, and mean squared error (MSE). We then compared these approaches in an applied example of an employment training program. The results indicate that combining full matching with double robust analysis performed best in both the simulations and the applied example, particularly when combined with machine learning estimation methods. To reduce bias, current guidelines advise researchers to select the technique with the best post-matching covariate balance, but this work finds that such an approach does not always minimize mean squared error (MSE). These findings have important implications for future research utilizing matching. To minimize MSE, investigators should consider additional diagnostics, and use of simulations tailored to the study of interest to identify the optimal matching and analysis combination.

Original languageEnglish (US)
Article number23222
JournalScientific Reports
Volume6
DOIs
StatePublished - Mar 16 2016

Fingerprint

Research Personnel
Guidelines
Education
Machine Learning

ASJC Scopus subject areas

  • General

Cite this

Colson, K. E., Rudolph, K. E., Zimmerman, S. C., Goin, D. E., Stuart, E., Laan, M. V. D., & Ahern, J. (2016). Optimizing matching and analysis combinations for estimating causal effects. Scientific Reports, 6, [23222]. https://doi.org/10.1038/srep23222

Optimizing matching and analysis combinations for estimating causal effects. / Colson, K. Ellicott; Rudolph, Kara E.; Zimmerman, Scott C.; Goin, Dana E.; Stuart, Elizabeth; Laan, Mark Van Der; Ahern, Jennifer.

In: Scientific Reports, Vol. 6, 23222, 16.03.2016.

Research output: Contribution to journalArticle

Colson, KE, Rudolph, KE, Zimmerman, SC, Goin, DE, Stuart, E, Laan, MVD & Ahern, J 2016, 'Optimizing matching and analysis combinations for estimating causal effects', Scientific Reports, vol. 6, 23222. https://doi.org/10.1038/srep23222
Colson, K. Ellicott ; Rudolph, Kara E. ; Zimmerman, Scott C. ; Goin, Dana E. ; Stuart, Elizabeth ; Laan, Mark Van Der ; Ahern, Jennifer. / Optimizing matching and analysis combinations for estimating causal effects. In: Scientific Reports. 2016 ; Vol. 6.
@article{107ff7d0204d4c28ad6728a84a7647fd,
title = "Optimizing matching and analysis combinations for estimating causal effects",
abstract = "Matching methods are common in studies across many disciplines. However, there is limited evidence on how to optimally combine matching with subsequent analysis approaches to minimize bias and maximize efficiency for the quantity of interest. We conducted simulations to compare the performance of a wide variety of matching methods and analysis approaches in terms of bias, variance, and mean squared error (MSE). We then compared these approaches in an applied example of an employment training program. The results indicate that combining full matching with double robust analysis performed best in both the simulations and the applied example, particularly when combined with machine learning estimation methods. To reduce bias, current guidelines advise researchers to select the technique with the best post-matching covariate balance, but this work finds that such an approach does not always minimize mean squared error (MSE). These findings have important implications for future research utilizing matching. To minimize MSE, investigators should consider additional diagnostics, and use of simulations tailored to the study of interest to identify the optimal matching and analysis combination.",
author = "Colson, {K. Ellicott} and Rudolph, {Kara E.} and Zimmerman, {Scott C.} and Goin, {Dana E.} and Elizabeth Stuart and Laan, {Mark Van Der} and Jennifer Ahern",
year = "2016",
month = "3",
day = "16",
doi = "10.1038/srep23222",
language = "English (US)",
volume = "6",
journal = "Scientific Reports",
issn = "2045-2322",
publisher = "Nature Publishing Group",

}

TY - JOUR

T1 - Optimizing matching and analysis combinations for estimating causal effects

AU - Colson, K. Ellicott

AU - Rudolph, Kara E.

AU - Zimmerman, Scott C.

AU - Goin, Dana E.

AU - Stuart, Elizabeth

AU - Laan, Mark Van Der

AU - Ahern, Jennifer

PY - 2016/3/16

Y1 - 2016/3/16

N2 - Matching methods are common in studies across many disciplines. However, there is limited evidence on how to optimally combine matching with subsequent analysis approaches to minimize bias and maximize efficiency for the quantity of interest. We conducted simulations to compare the performance of a wide variety of matching methods and analysis approaches in terms of bias, variance, and mean squared error (MSE). We then compared these approaches in an applied example of an employment training program. The results indicate that combining full matching with double robust analysis performed best in both the simulations and the applied example, particularly when combined with machine learning estimation methods. To reduce bias, current guidelines advise researchers to select the technique with the best post-matching covariate balance, but this work finds that such an approach does not always minimize mean squared error (MSE). These findings have important implications for future research utilizing matching. To minimize MSE, investigators should consider additional diagnostics, and use of simulations tailored to the study of interest to identify the optimal matching and analysis combination.

AB - Matching methods are common in studies across many disciplines. However, there is limited evidence on how to optimally combine matching with subsequent analysis approaches to minimize bias and maximize efficiency for the quantity of interest. We conducted simulations to compare the performance of a wide variety of matching methods and analysis approaches in terms of bias, variance, and mean squared error (MSE). We then compared these approaches in an applied example of an employment training program. The results indicate that combining full matching with double robust analysis performed best in both the simulations and the applied example, particularly when combined with machine learning estimation methods. To reduce bias, current guidelines advise researchers to select the technique with the best post-matching covariate balance, but this work finds that such an approach does not always minimize mean squared error (MSE). These findings have important implications for future research utilizing matching. To minimize MSE, investigators should consider additional diagnostics, and use of simulations tailored to the study of interest to identify the optimal matching and analysis combination.

UR - http://www.scopus.com/inward/record.url?scp=84961133604&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84961133604&partnerID=8YFLogxK

U2 - 10.1038/srep23222

DO - 10.1038/srep23222

M3 - Article

C2 - 26980444

AN - SCOPUS:84961133604

VL - 6

JO - Scientific Reports

JF - Scientific Reports

SN - 2045-2322

M1 - 23222

ER -