Counterfactual normalization: Proactively addressing dataset shift using causal mechanisms

Adarsh Subbaswamy, Suchi Saria

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Predictive models can fail to generalize from training to deployment environments because of dataset shift, posing a threat to model reliability in practice. As opposed to previous methods which use samples from the target distribution to reactively correct dataset shift, we propose using graphical knowledge of the causal mechanisms relating variables in a prediction problem to proactively remove variables that participate in spurious associations with the prediction target, allowing models to generalize across datasets. To accomplish this, we augment the causal graph with latent counter-factual variables that account for the underlying causal mechanisms, and show how we can estimate these variables. In our experiments we demonstrate that models using good estimates of the latent variables instead of the observed variables transfer better from training to target domains with minimal accuracy loss in the training domain.

Original languageEnglish (US)
Title of host publication34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018
EditorsRicardo Silva, Amir Globerson, Amir Globerson
PublisherAssociation For Uncertainty in Artificial Intelligence (AUAI)
Pages947-957
Number of pages11
Volume2
ISBN (Electronic)9781510871601
StatePublished - Jan 1 2018
Event34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018 - Monterey, United States
Duration: Aug 6 2018Aug 10 2018

Other

Other34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018
CountryUnited States
CityMonterey
Period8/6/188/10/18

Fingerprint

Experiments

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Subbaswamy, A., & Saria, S. (2018). Counterfactual normalization: Proactively addressing dataset shift using causal mechanisms. In R. Silva, A. Globerson, & A. Globerson (Eds.), 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018 (Vol. 2, pp. 947-957). Association For Uncertainty in Artificial Intelligence (AUAI).

Counterfactual normalization : Proactively addressing dataset shift using causal mechanisms. / Subbaswamy, Adarsh; Saria, Suchi.

34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018. ed. / Ricardo Silva; Amir Globerson; Amir Globerson. Vol. 2 Association For Uncertainty in Artificial Intelligence (AUAI), 2018. p. 947-957.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Subbaswamy, A & Saria, S 2018, Counterfactual normalization: Proactively addressing dataset shift using causal mechanisms. in R Silva, A Globerson & A Globerson (eds), 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018. vol. 2, Association For Uncertainty in Artificial Intelligence (AUAI), pp. 947-957, 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018, Monterey, United States, 8/6/18.
Subbaswamy A, Saria S. Counterfactual normalization: Proactively addressing dataset shift using causal mechanisms. In Silva R, Globerson A, Globerson A, editors, 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018. Vol. 2. Association For Uncertainty in Artificial Intelligence (AUAI). 2018. p. 947-957
Subbaswamy, Adarsh ; Saria, Suchi. / Counterfactual normalization : Proactively addressing dataset shift using causal mechanisms. 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018. editor / Ricardo Silva ; Amir Globerson ; Amir Globerson. Vol. 2 Association For Uncertainty in Artificial Intelligence (AUAI), 2018. pp. 947-957
@inproceedings{e41dd55c8ebc4731ad3b5e0828ad8a9d,
title = "Counterfactual normalization: Proactively addressing dataset shift using causal mechanisms",
abstract = "Predictive models can fail to generalize from training to deployment environments because of dataset shift, posing a threat to model reliability in practice. As opposed to previous methods which use samples from the target distribution to reactively correct dataset shift, we propose using graphical knowledge of the causal mechanisms relating variables in a prediction problem to proactively remove variables that participate in spurious associations with the prediction target, allowing models to generalize across datasets. To accomplish this, we augment the causal graph with latent counter-factual variables that account for the underlying causal mechanisms, and show how we can estimate these variables. In our experiments we demonstrate that models using good estimates of the latent variables instead of the observed variables transfer better from training to target domains with minimal accuracy loss in the training domain.",
author = "Adarsh Subbaswamy and Suchi Saria",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
volume = "2",
pages = "947--957",
editor = "Ricardo Silva and Amir Globerson and Amir Globerson",
booktitle = "34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018",
publisher = "Association For Uncertainty in Artificial Intelligence (AUAI)",

}

TY - GEN

T1 - Counterfactual normalization

T2 - Proactively addressing dataset shift using causal mechanisms

AU - Subbaswamy, Adarsh

AU - Saria, Suchi

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Predictive models can fail to generalize from training to deployment environments because of dataset shift, posing a threat to model reliability in practice. As opposed to previous methods which use samples from the target distribution to reactively correct dataset shift, we propose using graphical knowledge of the causal mechanisms relating variables in a prediction problem to proactively remove variables that participate in spurious associations with the prediction target, allowing models to generalize across datasets. To accomplish this, we augment the causal graph with latent counter-factual variables that account for the underlying causal mechanisms, and show how we can estimate these variables. In our experiments we demonstrate that models using good estimates of the latent variables instead of the observed variables transfer better from training to target domains with minimal accuracy loss in the training domain.

AB - Predictive models can fail to generalize from training to deployment environments because of dataset shift, posing a threat to model reliability in practice. As opposed to previous methods which use samples from the target distribution to reactively correct dataset shift, we propose using graphical knowledge of the causal mechanisms relating variables in a prediction problem to proactively remove variables that participate in spurious associations with the prediction target, allowing models to generalize across datasets. To accomplish this, we augment the causal graph with latent counter-factual variables that account for the underlying causal mechanisms, and show how we can estimate these variables. In our experiments we demonstrate that models using good estimates of the latent variables instead of the observed variables transfer better from training to target domains with minimal accuracy loss in the training domain.

UR - http://www.scopus.com/inward/record.url?scp=85058014446&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85058014446&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85058014446

VL - 2

SP - 947

EP - 957

BT - 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018

A2 - Silva, Ricardo

A2 - Globerson, Amir

A2 - Globerson, Amir

PB - Association For Uncertainty in Artificial Intelligence (AUAI)

ER -