Simulations evaluating resampling methods for causal discovery: ensemble performance and calibration
Causal discovery can be a powerful tool for investigating causality when a system can be observed but is inaccessible to experiments in practice. Despite this, it is rarely used in any scientific or medical fields. One of the major hurdles preventing the field of causal discovery from having a larger impact is that it is difficult to determine when the output of a causal discovery method can be trusted in a real-world setting. Trust is especially critical when human health is on the line. In this paper, we report the results of a series of simulation studies investigating the performance of different resampling methods as indicators of confidence in discovered graph features. We found that subsampling and sampling with replacement both performed surprisingly well, suggesting that they can serve as grounds for confidence in graph features. We also found that the calibration of subsampling and sampling with replacement had different convergence properties, suggesting that one's choice of which to use should depend on the sample size.
Code (0)
등록된 구현이 없습니다.
Tasks
Causal DiscoverySimilar Papers 제목 키워드 기반
An extensive simulation study evaluating the interaction of resampling techniques across multiple causal discovery contexts
Despite the accelerating presence of exploratory causal analysis in modern science and medicine, the available non-experimental methods for validating causal models are not well characterized. One of the most popular met…
Causal DiscoveryA Topological Sorting Criterion for Random Causal Directed Acyclic Graphs
Random directed acyclic graphs (DAGs) based on imposing an order on Erdős-Rényi and scale free random graphs are widely used for evaluating causal discovery algorithms. We show that in such DAGs, the set of nodes reachab…
Self-Compatibility: Evaluating Causal Discovery without Ground Truth
As causal ground truth is incredibly rare, causal discovery algorithms are commonly only evaluated on simulated data. This is concerning, given that simulations reflect preconceptions about generating processes regarding…
Causal DiscoveryModel SelectionEvaluating Temporal Observation-Based Causal Discovery Techniques Applied to Road Driver Behaviour
Autonomous robots are required to reason about the behaviour of dynamic agents in their environment. The creation of models to describe these relationships is typically accomplished through the application of causal disc…
Autonomous DrivingCausal DiscoveryTime Series AnalysisA Survey on Causal Discovery Methods for I.I.D. and Time Series Data
The ability to understand causality from data is one of the major milestones of human-level intelligence. Causal Discovery (CD) algorithms can identify the cause-effect relationships among the variables of a system from …
Causal DiscoveryTime Series