Causality-driven Hierarchical Structure Discovery for Reinforcement Learning
Hierarchical reinforcement learning (HRL) effectively improves agents' exploration efficiency on tasks with sparse reward, with the guide of high-quality hierarchical structures (e.g., subgoals or options). However, how to automatically discover high-quality hierarchical structures is still a great challenge. Previous HRL methods can hardly discover the hierarchical structures in complex environments due to the low exploration efficiency by exploiting the randomness-driven exploration paradigm. To address this issue, we propose CDHRL, a causality-driven hierarchical reinforcement learning framework, leveraging a causality-driven discovery instead of a randomness-driven exploration to effectively build high-quality hierarchical structures in complicated environments. The key insight is that the causalities among environment variables are naturally fit for modeling reachable subgoals and their dependencies and can perfectly guide to build high-quality hierarchical structures. The results in two complex environments, 2D-Minecraft and Eden, show that CDHRL significantly boosts exploration efficiency with the causality-driven paradigm.
Code (0)
등록된 구현이 없습니다.
Tasks
Hierarchical Reinforcement LearningMinecraftreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
From Query Tools to Causal Architects: Harnessing Large Language Models for Advanced Causal Discovery from Data
Large Language Models (LLMs) exhibit exceptional abilities for causal analysis between concepts in numerous societally impactful domains, including medicine, science, and law. Recent research on LLM performance in variou…
Causal DiscoveryScalable and Hybrid Ensemble-Based Causality Discovery
Causality discovery mines cause-effect relationships among different variables of a system and has been widely used in many disciplines including climatology and neuroscience. To discover causal relationships, many data-…
BenchmarkingDistributed ComputingTime SeriesTime Series AnalysisOrder-based Causal Discovery for Multistage Processes
Causality has become an increasingly important tool for gaining a deeper understanding of complex systems. Among various causal analysis methods, causal discovery, which identifies causal relationships among variables fr…
Computational EfficiencyEnhancing the Performance of Neural Networks Through Causal Discovery and Integration of Domain Knowledge
In this paper, we develop a generic methodology to encode hierarchical causality structure among observed variables into a neural network in order to improve its predictive performance. The proposed methodology, called c…
Causal DiscoverySystematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
Inducing causal relationships from observations is a classic problem in machine learning. Most work in causality starts from the premise that the causal variables themselves are observed. However, for AI agents such as r…
BenchmarkingCausal DiscoveryModel-based Reinforcement Learningreinforcement-learning+2