Self-Compatibility: Evaluating Causal Discovery without Ground Truth
As causal ground truth is incredibly rare, causal discovery algorithms are commonly only evaluated on simulated data. This is concerning, given that simulations reflect preconceptions about generating processes regarding noise distributions, model classes, and more. In this work, we propose a novel method for falsifying the output of a causal discovery algorithm in the absence of ground truth. Our key insight is that while statistical learning seeks stability across subsets of data points, causal learning should seek stability across subsets of variables. Motivated by this insight, our method relies on a notion of compatibility between causal graphs learned on different subsets of variables. We prove that detecting incompatibilities can falsify wrongly inferred causal relations due to violation of assumptions or errors from finite sample effects. Although passing such compatibility tests is only a necessary criterion for good performance, we argue that it provides strong evidence for the causal models whenever compatibility entails strong implications for the joint distribution. We also demonstrate experimentally that detection of incompatibilities can aid in causal model selection.
Code (1)
Tasks
Causal DiscoveryModel SelectionSimilar Papers 제목 키워드 기반
Evaluating Bivariate Causal Statements Based on Mutual Compatibility
For many real-world systems, causal ground truth is difficult to obtain, making claims about causal effects hard to assess. We develop methods for evaluating collections of $\binom{n}{2}$ bivariate causal statements over…
Neuropathic Pain Diagnosis Simulator for Causal Discovery Algorithm Evaluation
Discovery of causal relations from observational data is essential for many disciplines of science and real-world applications. However, unlike other machine learning algorithms, whose development has been greatly foster…
Causal DiscoverySelection biasCUTS: Neural Causal Discovery from Irregular Time-Series Data
Causal discovery from time-series data has been a central task in machine learning. Recently, Granger causality inference is gaining momentum due to its good explainability and high compatibility with emerging deep neura…
Causal DiscoveryGraph Neural NetworkIrregular Time SeriesTime Series+1Evaluating Temporal Observation-Based Causal Discovery Techniques Applied to Road Driver Behaviour
Autonomous robots are required to reason about the behaviour of dynamic agents in their environment. The creation of models to describe these relationships is typically accomplished through the application of causal disc…
Autonomous DrivingCausal DiscoveryTime Series AnalysisData Generating Process to Evaluate Causal Discovery Techniques for Time Series Data
Going beyond correlations, the understanding and identification of causal relationships in observational time series, an important subfield of Causal Discovery, poses a major challenge. The lack of access to a well-defin…
BenchmarkingCausal DiscoveryTime SeriesTime Series Analysis