Effect-Level Validation for Causal Discovery
Causal discovery is increasingly applied to large-scale telemetry data to estimate the effects of user-facing interventions, yet its reliability for decision-making in feedback-driven systems with strong self-selection remains unclear. In this paper, we propose an effect-centric, admissibility-first framework that treats discovered graphs as structural hypotheses and evaluates them by identifiability, stability, and falsification rather than by graph recovery accuracy alone. Empirically, we study the effect of early exposure to competitive gameplay on short-term retention using real-world game telemetry. We find that many statistically plausible discovery outputs do not admit point-identified causal queries once minimal temporal and semantic constraints are enforced, highlighting identifiability as a critical bottleneck for decision support. When identification is possible, several algorithm families converge to similar, decision-consistent effect estimates despite producing substantially different graph structures, including cases where the direct treatment-outcome edge is absent and the effect is preserved through indirect causal pathways. These converging estimates survive placebo, subsampling, and sensitivity refutation. In contrast, other methods exhibit sporadic admissibility and threshold-sensitive or attenuated effects due to endpoint ambiguity. These results suggest that graph-level metrics alone are inadequate proxies for causal reliability for a given target query. Therefore, trustworthy causal conclusions in telemetry-driven systems require prioritizing admissibility and effect-level validation over causal structural recovery alone.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Causal-based Framework for Multimodal Multivariate Time Series Validation Enhanced by Unsupervised Deep Learning as an Enabler for Industry 4.0
An advanced conceptual validation framework for multimodal multivariate time series defines a multi-level contextual anomaly detection ranging from an univariate context definition, to a multimodal abstract context repre…
Anomaly DetectionCausal DiscoveryContextual Anomaly DetectionRepresentation Learning+2Robust Time Series Causal Discovery for Agent-Based Model Validation
Agent-Based Model (ABM) validation is crucial as it helps ensuring the reliability of simulations, and causal discovery has become a powerful tool in this context. However, current causal discovery methods often face acc…
Causal DiscoveryTime SeriesFrom Local to Cluster: A Unified Framework for Causal Discovery with Latent Variables
Latent variables pose a fundamental challenge to causal discovery and inference. Conventional local methods focus on direct neighbors but fail to provide macro level insights. Cluster level methods enable macro causal re…
Computational EfficiencyCausal InferenceCausal Discovery with Attention-Based Convolutional Neural Networks
Having insight into the causal associations in a complex system facilitates decision making, e.g., for medical treatments, urban infrastructure improvements or financial investments. The amount of observational data grow…
Causal DiscoveryDecision MakingTime SeriesTime Series AnalysisCausal discovery of linear non-Gaussian acyclic models in the presence of latent confounders
Causal discovery from data affected by latent confounders is an important and difficult challenge. Causal functional model-based approaches have not been used to present variables whose relationships are affected by late…
Causal Discovery