Amortized Active Causal Induction with Deep Reinforcement Learning
We present Causal Amortized Active Structure Learning (CAASL), an active intervention design policy that can select interventions that are adaptive, real-time and that does not require access to the likelihood. This policy, an amortized network based on the transformer, is trained with reinforcement learning on a simulator of the design environment, and a reward function that measures how close the true causal graph is to a causal graph posterior inferred from the gathered data. On synthetic data and a single-cell gene expression simulator, we demonstrate empirically that the data acquired through our policy results in a better estimate of the underlying causal graph than alternative strategies. Our design policy successfully achieves amortized intervention design on the distribution of the training environment while also generalizing well to distribution shifts in test-time design environments. Further, our policy also demonstrates excellent zero-shot generalization to design environments with dimensionality higher than that during training, and to intervention types that it has not been trained on.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement Learningreinforcement-learningReinforcement LearningZero-shot GeneralizationSimilar Papers 제목 키워드 기반
Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
Inducing causal relationships from observations is a classic problem in machine learning. Most work in causality starts from the premise that the causal variables themselves are observed. However, for AI agents such as r…
BenchmarkingCausal DiscoveryModel-based Reinforcement Learningreinforcement-learning+2Learning Causal Overhypotheses through Exploration in Children and Computational Models
Despite recent progress in reinforcement learning (RL), RL algorithms for exploration still remain an active area of research. Existing methods often focus on state-based metrics, which do not consider the underlying cau…
Causal InferenceEfficient ExplorationReinforcement Learning (RL)Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data
On time-series data, most causal discovery methods fit a new model whenever they encounter samples from a new underlying causal graph. However, these samples often share relevant information which is lost when following …
Causal DiscoveryTime SeriesTime Series AnalysisDiscovering Causality for Efficient Cooperation in Multi-Agent Environments
In cooperative Multi-Agent Reinforcement Learning (MARL) agents are required to learn behaviours as a team to achieve a common goal. However, while learning a task, some agents may end up learning sub-optimal policies, n…
Causal DiscoveryMulti-agent Reinforcement LearningTheory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning
Learning transferable knowledge across similar but different settings is a fundamental component of generalized intelligence. In this paper, we approach the transfer learning challenge from a causal theory perspective. O…
Reinforcement LearningReinforcement Learning (RL)Transfer Learning