Learning to Interrupt: A Hierarchical Deep Reinforcement Learning Framework for Efficient Exploration

2018-07-30 · Tingguang Li, Jin Pan, Delong Zhu, Max Q. -H. Meng

To achieve scenario intelligence, humans must transfer knowledge to robots by developing goal-oriented algorithms, which are sometimes insensitive to dynamically changing environments. While deep reinforcement learning achieves significant success recently, it is still extremely difficult to be deployed in real robots directly. In this paper, we propose a hybrid structure named Option-Interruption in which human knowledge is embedded into a hierarchical reinforcement learning framework. Our architecture has two key components: options, represented by existing human-designed methods, can significantly speed up the training process and interruption mechanism, based on learnable termination functions, enables our system to quickly respond to the external environment. To implement this architecture, we derive a set of update rules based on policy gradient methods and present a complete training process. In the experiment part, our method is evaluated in Four-room navigation and exploration task, which shows the efficiency and flexibility of our framework.

📄 PDF Abstract BibTeX arXiv:1807.11150

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningEfficient ExplorationHierarchical Reinforcement LearningPolicy Gradient Methodsreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Virtuously Safe Reinforcement Learning

2018-05-29 · Henrik Aslund, El Mahdi El Mhamdi, Rachid Guerraoui, Alexandre Maurer

We show that when a third party, the adversary, steps into the two-party setting (agent and operator) of safely interruptible reinforcement learning, a trade-off has to be made between the probability of following the op…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Safe Exploration+1

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning

2022-10-13 · Shaohui Peng, Xing Hu, Rui Zhang, Ke Tang 외

Hierarchical reinforcement learning (HRL) effectively improves agents' exploration efficiency on tasks with sparse reward, with the guide of high-quality hierarchical structures (e.g., subgoals or options). However, how …

Hierarchical Reinforcement Learning Minecraft reinforcement-learning Reinforcement Learning+1

Hierarchical reinforcement learning for efficent exploration and transfer

2020-06-12 · ICML Workshop LifelongML 2020 7 · Lorenzo Steccanella, Simone Totaro, Damien Allonsius, Anders Jonsson

Sparse-reward domains are challenging for reinforcement learning algorithms since significant exploration is needed before encountering reward for the first time. Hierarchical reinforcement learning can facilitate explor…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical reinforcement learning for efficient exploration and transfer

2020-11-12 · Lorenzo Steccanella, Simone Totaro, Damien Allonsius, Anders Jonsson

Efficient Exploration Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning

2017-04-10 · NeurIPS 2017 12 · El Mahdi El Mhamdi, Rachid Guerraoui, Hadrien Hendrikx, Alexandre Maurer

In reinforcement learning, agents learn by performing actions and observing their outcomes. Sometimes, it is desirable for a human operator to \textit{interrupt} an agent in order to prevent dangerous situations from hap…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)