Multi-task Hierarchical Adversarial Inverse Reinforcement Learning
Multi-task Imitation Learning (MIL) aims to train a policy capable of performing a distribution of tasks based on multi-task expert demonstrations, which is essential for general-purpose robots. Existing MIL algorithms suffer from low data efficiency and poor performance on complex long-horizontal tasks. We develop Multi-task Hierarchical Adversarial Inverse Reinforcement Learning (MH-AIRL) to learn hierarchically-structured multi-task policies, which is more beneficial for compositional tasks with long horizons and has higher expert data efficiency through identifying and transferring reusable basic skills across tasks. To realize this, MH-AIRL effectively synthesizes context-based multi-task learning, AIRL (an IL approach), and hierarchical policy learning. Further, MH-AIRL can be adopted to demonstrations without the task or skill annotations (i.e., state-action pairs only) which are more accessible in practice. Theoretical justifications are provided for each module of MH-AIRL, and evaluations on challenging multi-task settings demonstrate superior performance and transferability of the multi-task policies learned with MH-AIRL as compared to SOTA MIL baselines.
Code (1)
Tasks
Imitation LearningMulti-Task Learningreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
SPLASH! Sample-efficient Preference-based inverse reinforcement learning for Long-horizon Adversarial tasks from Suboptimal Hierarchical demonstrations
Inverse Reinforcement Learning (IRL) presents a powerful paradigm for learning complex robotic tasks from human demonstrations. However, most approaches make the assumption that expert demonstrations are available, which…
Reinforcement LearningoIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions
Explicit engineering of reward functions for given environments has been a major hindrance to reinforcement learning methods. While Inverse Reinforcement Learning (IRL) is a solution to recover reward functions from demo…
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+2Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Control
Hierarchical Imitation Learning (HIL) has been proposed to recover highly-complex behaviors in long-horizon tasks from expert demonstrations by modeling the task hierarchy with the option framework. Existing methods eith…
Imitation LearningMulti-Task Learningreinforcement-learningReinforcement Learning+1HMPC-assisted Adversarial Inverse Reinforcement Learning for Smart Home Energy Management
This letter proposes an Adversarial Inverse Reinforcement Learning (AIRL)-based energy management method for a smart home, which incorporates an implicit thermal dynamics model. In the proposed method, historical optimal…
energy managementManagementModel Predictive Controlreinforcement-learning+1Off-Policy Adversarial Inverse Reinforcement Learning
Adversarial Imitation Learning (AIL) is a class of algorithms in Reinforcement learning (RL), which tries to imitate an expert without taking any reward from the environment and does not provide expert behavior directly …
continuous-controlContinuous ControlImitation Learningreinforcement-learning+3