Algorithmic causal structure emerging through compression
We explore the relationship between causality, symmetry, and compression. We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable. We propose a framework where causality emerges as a consequence of compressing data across multiple environments. We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold. We demonstrate how algorithmic causal and symmetric structures can emerge from minimizing upper bounds on Kolmogorov complexity, without knowledge of intervention targets. We hypothesize that these insights may also provide a novel perspective on the emergence of causality in machine learning models, such as large language models, where causal relationships may not be explicitly identifiable.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Binarized Neural Networks Converge Toward Algorithmic Simplicity: Empirical Support for the Learning-as-Compression Hypothesis
Understanding and controlling the informational complexity of neural networks is a central challenge in machine learning, with implications for generalization, optimization, and model capacity. While most approaches rely…
A note on the complexity of the causal ordering problem
In this note we provide a concise report on the complexity of the causal ordering problem, originally introduced by Simon to reason about causal dependencies implicit in systems of mathematical equations. We show that Si…
ManagementReal-Time Text Transmission via LLM-Based Entropy Coding over Fixed-Rate Channels
Learning, prediction, and compression are intimately connected: a model that accurately predicts the next symbol in a sequence can be coupled with a source coder to compress that sequence near its information-theoretic l…
Quantum causal inference in the presence of hidden common causes: An entropic approach
Quantum causality is an emerging field of study which has the potential to greatly advance our understanding of quantum systems. In this paper, we put forth a theoretical framework for merging quantum information science…
Causal InferenceGranger Causal Structure Reconstruction from Heterogeneous Multivariate Time Series
Granger causal structure reconstruction is an emerging topic that can uncover causal relationship behind multivariate time series data. In many real-world systems, it is common to encounter a large amount of multivariate…
Time SeriesTime Series Analysis