Learning When and What to Ask: a Hierarchical Reinforcement Learning Framework

2021-09-29 · Khanh Xuan Nguyen, Yonatan Bisk, Hal Daumé III

Reliable AI agents should be mindful of the limits of their knowledge and consult humans when sensing that they do not have sufficient knowledge to make sound decisions. We formulate a hierarchical reinforcement learning framework for learning to decide when to request additional information from humans and what type of information would be helpful to request. Our framework extends partially-observed Markov decision processes (POMDPs) by allowing an agent to interact with an assistant to leverage their knowledge in accomplishing tasks. Results on a simulated human-assisted navigation problem demonstrate the effectiveness of our framework: aided with an interaction policy learned by our method, a navigation policy achieves up to a 7× improvement in task success rate compared to performing tasks only by itself. The interaction policy is also efficient: on average, only a quarter of all actions taken during a task execution are requests for information. We analyze benefits and challenges of learning with a hierarchical policy structure and suggest directions for future work.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Hierarchical Reinforcement Learning Framework for Stochastic Spaceflight Campaign Design

2021-03-16 · Yuji Takubo, Hao Chen, Koki Ho

This paper develops a hierarchical reinforcement learning architecture for multimission spaceflight campaign design under uncertainty, including vehicle design, infrastructure deployment planning, and space transportatio…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Hierarchical Reinforcement Learning with Timed Subgoals

2021-12-06 · NeurIPS 2021 12 · Nico Gürtler, Dieter Büchler, Georg Martius

Hierarchical reinforcement learning (HRL) holds great potential for sample-efficient learning on challenging long-horizon tasks. In particular, letting a higher level assign subgoals to a lower level has been shown to en…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Possibility Before Utility: Learning And Using Hierarchical Affordances

2022-03-23 · ICLR 2022 4 · Robby Costales, Shariq Iqbal, Fei Sha

Reinforcement learning algorithms struggle on tasks with complex hierarchical dependency structures. Humans and other intelligent agents do not waste time assessing the utility of every high-level action in existence, bu…

Hierarchical Reinforcement Learning Navigate reinforcement-learning Reinforcement Learning+1

Reinforcement Learning with Options and State Representation

2024-03-16 · Ayoub Ghriss, Masashi Sugiyama, Alessandro Lazaric

The current thesis aims to explore the reinforcement learning field and build on existing methods to produce improved ones to tackle the problem of learning in high-dimensional and complex environments. It addresses such…

Decision Making Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning

2017-12-20 · ICLR 2018 1 · Tianmin Shu, Caiming Xiong, Richard Socher

Learning policies for complex tasks that require multiple different skills is a major challenge in reinforcement learning (RL). It is also a requirement for its deployment in real-world scenarios. This paper proposes a n…

Minecraft reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)