Learning World Graphs to Accelerate Hierarchical Reinforcement Learning

2019-07-01 · Wenling Shang, Alex Trott, Stephan Zheng, Caiming Xiong, Richard Socher

In many real-world scenarios, an autonomous agent often encounters various tasks within a single complex environment. We propose to build a graph abstraction over the environment structure to accelerate the learning of these tasks. Here, nodes are important points of interest (pivotal states) and edges represent feasible traversals between them. Our approach has two stages. First, we jointly train a latent pivotal state model and a curiosity-driven goal-conditioned policy in a task-agnostic manner. Second, provided with the information from the world graph, a high-level Manager quickly finds solution to new tasks and expresses subgoals in reference to pivotal states to a low-level Worker. The Worker can then also leverage the graph to easily traverse to the pivotal states of interest, even across long distance, and explore non-locally. We perform a thorough ablation study to evaluate our approach on a suite of challenging maze tasks, demonstrating significant advantages from the proposed framework over baselines that lack world graph knowledge in terms of performance and efficiency.

📄 PDF Abstract BibTeX arXiv:1907.00664

Code (0)

등록된 구현이 없습니다.

Tasks

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning World Graph Decompositions To Accelerate Reinforcement Learning

2019-09-25 · Wenling Shang, Alex Trott, Stephan Zheng, Caiming Xiong 외

Efficiently learning to solve tasks in complex environments is a key challenge for reinforcement learning (RL) agents. We propose to decompose a complex environment using a task-agnostic world graphs, an abstraction tha…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks

2022-07-25 · Laura Stops, Roel Leenhouts, Qinghe Gao, Artur M. Schweidtmann

Process synthesis experiences a disruptive transformation accelerated by digitization and artificial intelligence. We propose a reinforcement learning algorithm for chemical process design based on a state-of-the-art act…

Chemical Process Decision Making Hierarchical Reinforcement Learning reinforcement-learning+2

DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning

2024-06-16 · Utsav Singh, Souradip Chakraborty, Wesley A. Suttle, Brian M. Sadler 외

Learning control policies to perform complex robotics tasks from human preference data presents significant challenges. On the one hand, the complexity of such tasks typically requires learning policies to perform a vari…

Computational Efficiency Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Combining imagination and heuristics to learn strategies that generalize

2018-09-10 · Erik J Peterson, Necati Alp Müyesser, Timothy Verstynen, Kyle Dunovan

Deep reinforcement learning can match or exceed human performance in stable contexts, but with minor changes to the environment artificial networks, unlike humans, often cannot adapt. Humans rely on a combination of heur…

Deep Reinforcement Learning Hierarchical Reinforcement Learning Position reinforcement-learning+2

Deep Hierarchical Reinforcement Learning Based Recommendations via Multi-goals Abstraction

2019-03-22 · Dongyang Zhao, Liang Zhang, Bo Zhang, Lizhou Zheng 외

The recommender system is an important form of intelligent application, which assists users to alleviate from information redundancy. Among the metrics used to evaluate a recommender system, the metric of conversion has …

Hierarchical Reinforcement Learning Recommendation Systems reinforcement-learning Reinforcement Learning+1