Applications of Common Entropy for Causal Inference
We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent. The minimum entropy required for such a latent is known as common entropy in information theory. We extend this notion to Renyi common entropy by minimizing the Renyi entropy of the latent variable. To efficiently compute common entropy, we propose an iterative algorithm that can be used to discover the trade-off between the entropy of the latent variable and the conditional mutual information of the observed variables. We show two applications of common entropy in causal inference: First, under the assumption that there are no low-entropy mediators, it can be used to distinguish causation from spurious correlation among almost all joint distributions on simple causal graphs with two observed variables. Second, common entropy can be used to improve constraint-based methods such as PC or FCI algorithms in the small-sample regime, where these methods are known to struggle. We propose a modification to these constraint-based methods to assess if a separating set found by these algorithms is valid using common entropy. We finally evaluate our algorithms on synthetic and real data to establish their performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Causal InferencevalidMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variable-lag Granger Causality and Transfer Entropy for Time Series Analysis
Granger causality is a fundamental technique for causal inference in time series data, commonly used in the social and biological sciences. Typical operationalizations of Granger causality make a strong assumption that e…
Causal InferenceDynamic Time WarpingLeadership InferenceTime Series+1Quantum causal inference in the presence of hidden common causes: An entropic approach
Quantum causality is an emerging field of study which has the potential to greatly advance our understanding of quantum systems. In this paper, we put forth a theoretical framework for merging quantum information science…
Causal InferenceEntropic Causal Inference
We consider the problem of identifying the causal direction between two discrete random variables using observational data. Unlike previous work, we keep the most general functional model but make an assumption on the un…
Causal InferenceOn Geometry of Information Flow for Causal Inference
Causal inference is perhaps one of the most fundamental concepts in science, beginning originally from the works of some of the ancient philosophers, through today, but also weaved strongly in current work from statistic…
Causal InferenceEnd-to-End Balancing for Causal Continuous Treatment-Effect Estimation
We study the problem of observational causal inference with continuous treatments in the framework of inverse propensity-score weighting. To obtain stable weights, we design a new algorithm based on entropy balancing tha…
Causal Inference