Human-Interactive Subgoal Supervision for Efficient Inverse Reinforcement Learning

2018-06-22 · Xinlei Pan, Eshed Ohn-Bar, Nicholas Rhinehart, Yan Xu, Yilin Shen, Kris M. Kitani

Humans are able to understand and perform complex tasks by strategically structuring the tasks into incremental steps or subgoals. For a robot attempting to learn to perform a sequential task with critical subgoal states, such states can provide a natural opportunity for interaction with a human expert. This paper analyzes the benefit of incorporating a notion of subgoals into Inverse Reinforcement Learning (IRL) with a Human-In-The-Loop (HITL) framework. The learning process is interactive, with a human expert first providing input in the form of full demonstrations along with some subgoal states. These subgoal states define a set of subtasks for the learning agent to complete in order to achieve the final goal. The learning agent queries for partial demonstrations corresponding to each subtask as needed when the agent struggles with the subtask. The proposed Human Interactive IRL (HI-IRL) framework is evaluated on several discrete path-planning tasks. We demonstrate that subgoal-based interactive structuring of the learning task results in significantly more efficient learning, requiring only a fraction of the demonstration data needed for learning the underlying reward function with the baseline IRL model.

📄 PDF Abstract BibTeX arXiv:1806.08479

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Curricular Subgoals for Inverse Reinforcement Learning

2023-06-14 · Shunyu Liu, Yunpeng Qing, Shuqi Xu, Hongyan Wu 외

Inverse Reinforcement Learning (IRL) aims to reconstruct the reward function from expert demonstrations to facilitate policy learning, and has demonstrated its remarkable success in imitation learning. To promote expert-…

Autonomous Driving D4RL Imitation Learning reinforcement-learning+1

Advancing Autonomous VLM Agents via Variational Subgoal-Conditioned Reinforcement Learning

2025-02-11 · Qingyuan Wu, Jianheng Liu, Jianye Hao, Jun Wang 외

State-of-the-art (SOTA) reinforcement learning (RL) methods have enabled vision-language model (VLM) agents to learn from interaction with online environments without human supervision. However, these methods often strug…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Can Co-robots Learn to Teach?

2016-11-22 · Harshal Maske, Emily Kieson, Girish Chowdhary, Charles Abramson

We explore beyond existing work on learning from demonstration by asking the question: Can robots learn to teach?, that is, can a robot autonomously learn an instructional policy from expert demonstration and use it to i…

Clustering Reinforcement Learning

Subgoal-based Reward Shaping to Improve Efficiency in Reinforcement Learning

2021-04-13 · Takato Okudo, Seiji Yamada

Reinforcement learning, which acquires a policy maximizing long-term rewards, has been actively studied. Unfortunately, this learning type is too slow and difficult to use in practical situations because the state-action…

AI Agent reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

HRL4IN: Hierarchical Reinforcement Learning for Interactive Navigation with Mobile Manipulators

2019-10-24 · Chengshu Li, Fei Xia, Roberto Martin-Martin, Silvio Savarese

Most common navigation tasks in human environments require auxiliary arm interactions, e.g. opening doors, pressing buttons and pushing obstacles away. This type of navigation tasks, which we call Interactive Navigation,…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)