Reinforcement Causal Structure Learning on Order Graph
Learning directed acyclic graph (DAG) that describes the causality of observed data is a very challenging but important task. Due to the limited quantity and quality of observed data, and non-identifiability of causal graph, it is almost impossible to infer a single precise DAG. Some methods approximate the posterior distribution of DAGs to explore the DAG space via Markov chain Monte Carlo (MCMC), but the DAG space is over the nature of super-exponential growth, accurately characterizing the whole distribution over DAGs is very intractable. In this paper, we propose {Reinforcement Causal Structure Learning on Order Graph} (RCL-OG) that uses order graph instead of MCMC to model different DAG topological orderings and to reduce the problem size. RCL-OG first defines reinforcement learning with a new reward mechanism to approximate the posterior distribution of orderings in an efficacy way, and uses deep Q-learning to update and transfer rewards between nodes. Next, it obtains the probability transition model of nodes on order graph, and computes the posterior probability of different orderings. In this way, we can sample on this model to obtain the ordering with high probability. Experiments on synthetic and benchmark datasets show that RCL-OG provides accurate posterior probability approximation and achieves better results than competitive causal discovery algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
Causal DiscoveryQ-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Novel Ordering-based Approaches for Causal Structure Learning in the Presence of Unobserved Variables
We propose ordering-based approaches for learning the maximal ancestral graph (MAG) of a structural equation model (SEM) up to its Markov equivalence class (MEC) in the presence of unobserved variables. Existing ordering…
Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
Inducing causal relationships from observations is a classic problem in machine learning. Most work in causality starts from the premise that the causal variables themselves are observed. However, for AI agents such as r…
BenchmarkingCausal DiscoveryModel-based Reinforcement Learningreinforcement-learning+2CORE: Towards Scalable and Efficient Causal Discovery with Reinforcement Learning
Causal discovery is the challenging task of inferring causal structure from data. Motivated by Pearl's Causal Hierarchy (PCH), which tells us that passive observations alone are not enough to distinguish correlation from…
Causal DiscoveryDeep Reinforcement Learningreinforcement-learningReinforcement LearningCausal Reasoning from Meta-reinforcement Learning
Discovering and exploiting the causal structure in the environment is a crucial challenge for intelligent agents. Here we explore whether causal reasoning can emerge via meta-reinforcement learning. We train a recurrent …
counterfactualMeta Reinforcement Learningreinforcement-learningReinforcement Learning+1Causal Structure Learning: a Bayesian approach based on random graphs
A Random Graph is a random object which take its values in the space of graphs. We take advantage of the expressibility of graphs in order to model the uncertainty about the existence of causal relationships within a giv…