Estimating the effect of treatment on binary outcomes using full matching on the propensity score

Peter C. Austin, Elizabeth Stuart

Research output: Contribution to journalArticle

Abstract

Many non-experimental studies use propensity-score methods to estimate causal effects by balancing treatment and control groups on a set of observed baseline covariates. Full matching on the propensity score has emerged as a particularly effective and flexible method for utilizing all available data, and creating well-balanced treatment and comparison groups. However, full matching has been used infrequently with binary outcomes, and relatively little work has investigated the performance of full matching when estimating effects on binary outcomes. This paper describes methods that can be used for estimating the effect of treatment on binary outcomes when using full matching. It then used Monte Carlo simulations to evaluate the performance of these methods based on full matching (with and without a caliper), and compared their performance with that of nearest neighbour matching (with and without a caliper) and inverse probability of treatment weighting. The simulations varied the prevalence of the treatment and the strength of association between the covariates and treatment assignment. Results indicated that all of the approaches work well when the strength of confounding is relatively weak. With stronger confounding, the relative performance of the methods varies, with nearest neighbour matching with a caliper showing consistently good performance across a wide range of settings. We illustrate the approaches using a study estimating the effect of inpatient smoking cessation counselling on survival following hospitalization for a heart attack.

Original languageEnglish (US)
Pages (from-to)2505-2525
Number of pages21
JournalStatistical Methods in Medical Research
Volume26
Issue number6
DOIs
StatePublished - Dec 1 2017

Fingerprint

Propensity Score
Binary Outcomes
Confounding
Covariates
Nearest Neighbor
Therapeutics
Causal Effect
Smoking
Smoking Cessation
Balancing
Weighting
Counseling
Inpatients
Baseline
Hospitalization
Assignment
Monte Carlo Simulation
Myocardial Infarction
Attack
Vary

Keywords

  • bias
  • full matching
  • inverse probability of treatment weighting
  • matching
  • Monte Carlo simulations
  • observational studies
  • Propensity score

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability
  • Health Information Management

Cite this

Estimating the effect of treatment on binary outcomes using full matching on the propensity score. / Austin, Peter C.; Stuart, Elizabeth.

In: Statistical Methods in Medical Research, Vol. 26, No. 6, 01.12.2017, p. 2505-2525.

Research output: Contribution to journalArticle

@article{b8f412ce194e40c09e55b68e0848153f,
title = "Estimating the effect of treatment on binary outcomes using full matching on the propensity score",
abstract = "Many non-experimental studies use propensity-score methods to estimate causal effects by balancing treatment and control groups on a set of observed baseline covariates. Full matching on the propensity score has emerged as a particularly effective and flexible method for utilizing all available data, and creating well-balanced treatment and comparison groups. However, full matching has been used infrequently with binary outcomes, and relatively little work has investigated the performance of full matching when estimating effects on binary outcomes. This paper describes methods that can be used for estimating the effect of treatment on binary outcomes when using full matching. It then used Monte Carlo simulations to evaluate the performance of these methods based on full matching (with and without a caliper), and compared their performance with that of nearest neighbour matching (with and without a caliper) and inverse probability of treatment weighting. The simulations varied the prevalence of the treatment and the strength of association between the covariates and treatment assignment. Results indicated that all of the approaches work well when the strength of confounding is relatively weak. With stronger confounding, the relative performance of the methods varies, with nearest neighbour matching with a caliper showing consistently good performance across a wide range of settings. We illustrate the approaches using a study estimating the effect of inpatient smoking cessation counselling on survival following hospitalization for a heart attack.",
keywords = "bias, full matching, inverse probability of treatment weighting, matching, Monte Carlo simulations, observational studies, Propensity score",
author = "Austin, {Peter C.} and Elizabeth Stuart",
year = "2017",
month = "12",
day = "1",
doi = "10.1177/0962280215601134",
language = "English (US)",
volume = "26",
pages = "2505--2525",
journal = "Statistical Methods in Medical Research",
issn = "0962-2802",
publisher = "SAGE Publications Ltd",
number = "6",

}

TY - JOUR

T1 - Estimating the effect of treatment on binary outcomes using full matching on the propensity score

AU - Austin, Peter C.

AU - Stuart, Elizabeth

PY - 2017/12/1

Y1 - 2017/12/1

N2 - Many non-experimental studies use propensity-score methods to estimate causal effects by balancing treatment and control groups on a set of observed baseline covariates. Full matching on the propensity score has emerged as a particularly effective and flexible method for utilizing all available data, and creating well-balanced treatment and comparison groups. However, full matching has been used infrequently with binary outcomes, and relatively little work has investigated the performance of full matching when estimating effects on binary outcomes. This paper describes methods that can be used for estimating the effect of treatment on binary outcomes when using full matching. It then used Monte Carlo simulations to evaluate the performance of these methods based on full matching (with and without a caliper), and compared their performance with that of nearest neighbour matching (with and without a caliper) and inverse probability of treatment weighting. The simulations varied the prevalence of the treatment and the strength of association between the covariates and treatment assignment. Results indicated that all of the approaches work well when the strength of confounding is relatively weak. With stronger confounding, the relative performance of the methods varies, with nearest neighbour matching with a caliper showing consistently good performance across a wide range of settings. We illustrate the approaches using a study estimating the effect of inpatient smoking cessation counselling on survival following hospitalization for a heart attack.

AB - Many non-experimental studies use propensity-score methods to estimate causal effects by balancing treatment and control groups on a set of observed baseline covariates. Full matching on the propensity score has emerged as a particularly effective and flexible method for utilizing all available data, and creating well-balanced treatment and comparison groups. However, full matching has been used infrequently with binary outcomes, and relatively little work has investigated the performance of full matching when estimating effects on binary outcomes. This paper describes methods that can be used for estimating the effect of treatment on binary outcomes when using full matching. It then used Monte Carlo simulations to evaluate the performance of these methods based on full matching (with and without a caliper), and compared their performance with that of nearest neighbour matching (with and without a caliper) and inverse probability of treatment weighting. The simulations varied the prevalence of the treatment and the strength of association between the covariates and treatment assignment. Results indicated that all of the approaches work well when the strength of confounding is relatively weak. With stronger confounding, the relative performance of the methods varies, with nearest neighbour matching with a caliper showing consistently good performance across a wide range of settings. We illustrate the approaches using a study estimating the effect of inpatient smoking cessation counselling on survival following hospitalization for a heart attack.

KW - bias

KW - full matching

KW - inverse probability of treatment weighting

KW - matching

KW - Monte Carlo simulations

KW - observational studies

KW - Propensity score

UR - http://www.scopus.com/inward/record.url?scp=85038880269&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038880269&partnerID=8YFLogxK

U2 - 10.1177/0962280215601134

DO - 10.1177/0962280215601134

M3 - Article

C2 - 26329750

AN - SCOPUS:85038880269

VL - 26

SP - 2505

EP - 2525

JO - Statistical Methods in Medical Research

JF - Statistical Methods in Medical Research

SN - 0962-2802

IS - 6

ER -