Active Learning of Causal Structures with Deep Reinforcement Learning
We study the problem of experiment design to learn causal structures from interventional data. We consider an active learning setting in which the experimenter decides to intervene on one of the variables in the system in each step and uses the results of the intervention to recover further causal relationships among the variables. The goal is to fully identify the causal structures with minimum number of interventions. We present the first deep reinforcement learning based solution for the problem of experiment design. In the proposed method, we embed input graphs to vectors using a graph neural network and feed them to another neural network which outputs a variable for performing intervention in each step. Both networks are trained jointly via a Q-iteration algorithm. Experimental results show that the proposed method achieves competitive performance in recovering causal structures with respect to previous works, while significantly reducing execution time in dense graphs.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningDeep Reinforcement LearningGraph Neural Networkreinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CORE: Towards Scalable and Efficient Causal Discovery with Reinforcement Learning
Causal discovery is the challenging task of inferring causal structure from data. Motivated by Pearl's Causal Hierarchy (PCH), which tells us that passive observations alone are not enough to distinguish correlation from…
Causal DiscoveryDeep Reinforcement Learningreinforcement-learningReinforcement LearningLearning Causal Overhypotheses through Exploration in Children and Computational Models
Despite recent progress in reinforcement learning (RL), RL algorithms for exploration still remain an active area of research. Existing methods often focus on state-based metrics, which do not consider the underlying cau…
Causal InferenceEfficient ExplorationReinforcement Learning (RL)Causality-driven Hierarchical Structure Discovery for Reinforcement Learning
Hierarchical reinforcement learning (HRL) effectively improves agents' exploration efficiency on tasks with sparse reward, with the guide of high-quality hierarchical structures (e.g., subgoals or options). However, how …
Hierarchical Reinforcement LearningMinecraftreinforcement-learningReinforcement Learning+1Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning?
Causal confusion is a phenomenon where an agent learns a policy that reflects imperfect spurious correlations in the data. Such a policy may falsely appear to be optimal during training if most of the training data conta…
reinforcement-learningReinforcement LearningReinforcement Learning is not a Causal problem
We use an analogy between non-isomorphic mathematical structures defined over the same set and the algebras induced by associative and causal levels of information in order to argue that Reinforcement Learning, in its cu…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)