Exploring Hierarchy-Aware Inverse Reinforcement Learning

2018-07-13 · Chris Cundy, Daniel Filan

We introduce a new generative model for human planning under the Bayesian Inverse Reinforcement Learning (BIRL) framework which takes into account the fact that humans often plan using hierarchical strategies. We describe the Bayesian Inverse Hierarchical RL (BIHRL) algorithm for inferring the values of hierarchical planners, and use an illustrative toy model to show that BIHRL retains accuracy where standard BIRL fails. Furthermore, BIHRL is able to accurately predict the goals of `Wikispeedia' game players, with inclusion of hierarchical structure in the model resulting in a large boost in accuracy. We show that BIHRL is able to significantly outperform BIRL even when we only have a weak prior on the hierarchical structure of the plans available to the agent, and discuss the significant challenges that remain for scaling up this framework to more realistic settings.

📄 PDF Abstract BibTeX arXiv:1807.05037

Code (0)

등록된 구현이 없습니다.

Tasks

BIRL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?

2019-09-23 · Ofir Nachum, Haoran Tang, Xingyu Lu, Shixiang Gu 외

Hierarchical reinforcement learning has demonstrated significant success at solving difficult reinforcement learning (RL) tasks. Previous works have motivated the use of hierarchy by appealing to a number of intuitive be…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Control

2022-10-05 · Jiayu Chen, Tian Lan, Vaneet Aggarwal

Hierarchical Imitation Learning (HIL) has been proposed to recover highly-complex behaviors in long-horizon tasks from expert demonstrations by modeling the task hierarchy with the option framework. Existing methods eith…

Imitation Learning Multi-Task Learning reinforcement-learning Reinforcement Learning+1

Neuroevolution-Based Inverse Reinforcement Learning

2016-08-09 · Karan K. Budhraja, Tim Oates

The problem of Learning from Demonstration is targeted at learning to perform tasks based on observed examples. One approach to Learning from Demonstration is Inverse Reinforcement Learning, in which actions are observed…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Context-Hierarchy Inverse Reinforcement Learning

2022-02-25 · Wei Gao, David Hsu, Wee Sun Lee

An inverse reinforcement learning (IRL) agent learns to act intelligently by observing expert demonstrations and learning the expert's underlying reward function. Although learning the reward functions from demonstration…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Estimating cognitive biases with attention-aware inverse planning

2025-10-29 · Sounak Banerjee, Daphne Cornelisse, Deepak Gopinath, Emily Sumner 외 arxiv

People's goal-directed behaviors are influenced by their cognitive biases, and autonomous systems that interact with people should be aware of this. For example, people's attention to objects in their environment will be…

Reinforcement Learning