Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation
We study the problem of model selection in causal inference, specifically for conditional average treatment effect (CATE) estimation. Unlike machine learning, there is no perfect analogue of cross-validation for model selection as we do not observe the counterfactual potential outcomes. Towards this, a variety of surrogate metrics have been proposed for CATE model selection that use only observed data. However, we do not have a good understanding regarding their effectiveness due to limited comparisons in prior studies. We conduct an extensive empirical analysis to benchmark the surrogate model selection metrics introduced in the literature, as well as the novel ones introduced in this work. We ensure a fair comparison by tuning the hyperparameters associated with these metrics via AutoML, and provide more detailed trends by incorporating realistic datasets via generative modeling. Our analysis suggests novel model selection strategies based on careful hyperparameter selection of CATE estimators and causal ensembling.
Code (1)
Tasks
AutoMLCausal InferencecounterfactualModel SelectionSimilar Papers 제목 키워드 기반
Improving Data-driven Heterogeneous Treatment Effect Estimation Under Structure Uncertainty
Estimating how a treatment affects units individually, known as heterogeneous treatment effect (HTE) estimation, is an essential part of decision-making and policy implementation. The accumulation of large amounts of dat…
Decision Makingfeature selectionHeterogeneous Treatment Effect EstimationA Large-Scale Empirical Comparison of Meta-Learners and Causal Forests for Heterogeneous Treatment Effect Estimation in Marketing Uplift Modeling
Estimating Conditional Average Treatment Effects (CATE) at the individual level is central to precision marketing, yet systematic benchmarking of uplift modeling methods at industrial scale remains limited. We present Up…
Heterogeneous Treatment Effect EstimationModified Causal Forests for Estimating Heterogeneous Causal Effects
Uncovering the heterogeneity of causal effects of policies and business decisions at various levels of granularity provides substantial value to decision makers. This paper develops new estimation and inference procedure…
Treatment Effect Heterogeneity in Regression Discontinuity Designs
Empirical studies using Regression Discontinuity (RD) designs often explore heterogeneous treatment effects based on pretreatment covariates. However, the lack of formal statistical methods has led to the widespread use …
regressionEstimating Heterogeneous Causal Effect of Polysubstance Usage on Drug Overdose from Large-Scale Electronic Health Record
Drug overdose has become a public health crisis in the United States with devastating consequences. However, most of the drug overdose incidences are the consequence of recitative polysubstance usage over a defined perio…
Causal Inference