Environment Design for Inverse Reinforcement Learning
Learning a reward function from demonstrations suffers from low sample-efficiency. Even with abundant data, current inverse reinforcement learning methods that focus on learning from a single environment can fail to handle slight changes in the environment dynamics. We tackle these challenges through adaptive environment design. In our framework, the learner repeatedly interacts with the expert, with the former selecting environments to identify the reward function as quickly as possible from the expert's demonstrations in said environments. This results in improvements in both sample-efficiency and robustness, as we show experimentally, for both exact and approximate inference.
Code (1)
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Option Compatible Reward Inverse Reinforcement Learning
Reinforcement learning in complex environments is a challenging problem. In particular, the success of reinforcement learning algorithms depends on a well-designed reward function. Inverse reinforcement learning (IRL) so…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Transfer LearningInverse reinforcement learning for video games
Deep reinforcement learning achieves superhuman performance in a range of video game environments, but requires that a designer manually specify a reward function. It is often easier to provide demonstrations of a target…
continuous-controlContinuous ControlDeep Reinforcement Learningreinforcement-learning+2Extrinsicaly Rewarded Soft Q Imitation Learning with Discriminator
Imitation learning is often used in addition to reinforcement learning in environments where reward design is difficult or where the reward is sparse, but it is difficult to be able to imitate well in unknown states from…
Imitation LearningMuJoCoQ-Learningreinforcement-learning+1oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions
Explicit engineering of reward functions for given environments has been a major hindrance to reinforcement learning methods. While Inverse Reinforcement Learning (IRL) is a solution to recover reward functions from demo…
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+2Demonstration-efficient Inverse Reinforcement Learning in Procedurally Generated Environments
Deep Reinforcement Learning achieves very good results in domains where reward functions can be manually engineered. At the same time, there is growing interest within the community in using games based on Procedurally C…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)