An evaluation framework for comparing causal inference models
Estimation of causal effects is the core objective of many scientific disciplines. However, it remains a challenging task, especially when the effects are estimated from observational data. Recently, several promising machine learning models have been proposed for causal effect estimation. The evaluation of these models has been based on the mean values of the error of the Average Treatment Effect (ATE) as well as of the Precision in Estimation of Heterogeneous Effect (PEHE). In this paper, we propose to complement the evaluation of causal inference models using concrete statistical evidence, including the performance profiles of Dolan and Mor{\'e}, as well as non-parametric and post-hoc statistical tests. The main motivation behind this approach is the elimination of the influence of a small number of instances or simulation on the benchmarking process, which in some cases dominate the results. We use the proposed evaluation methodology to compare several state-of-the-art causal effect estimation models.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingCausal InferenceSimilar Papers 제목 키워드 기반
Integrating Unstructured Text into Causal Inference: Empirical Evidence from Real Data
Causal inference, a critical tool for informing business decisions, traditionally relies heavily on structured data. However, in many real-world scenarios, such data can be incomplete or unavailable. This paper presents …
Causal InferenceHow and Why to Use Experimental Data to Evaluate Methods for Observational Causal Inference
Methods that infer causal dependence from observational data are central to many areas of science, including medicine, economics, and the social sciences. A variety of theoretical properties of these methods have been pr…
Causal InferenceA primer on optimal transport for causal inference with observational data
The theory of optimal transportation has developed into a powerful and elegant framework for comparing probability distributions, with wide-ranging applications in all areas of science. The fundamental idea of analyzing …
Causal InferencecounterfactualEconometricsAnalysis of cause-effect inference by comparing regression errors
We address the problem of inferring the causal direction between two variables by comparing the least-squares errors of the predictions in both possible directions. Under the assumption of an independence between the fun…
Causal InferenceregressionHigh Precision Causal Model Evaluation with Conditional Randomization
The gold standard for causal model evaluation involves comparing model predictions with true effects estimated from randomized controlled trials (RCT). However, RCTs are not always feasible or ethical to perform. In cont…
Causal Inferencemodel