Variational Imitation Learning with Diverse-quality Demonstrations
Learning from demonstrations can be challenging when the quality of demonstrations is diverse, and even more so when the quality is unknown and there is no additional information to estimate the quality. We propose a new method for imitation learning in such scenarios. We show that simple quality-estimation approaches might fail due to compounding error, and fix this issue by jointly estimating both the quality and reward using a variational approach. Our method is easy to implement within reinforcement-learning frameworks and also achieves state-of-the-art performance on continuous-control benchmarks.Our work enables scalable and data-efficient imitation learning under more realistic settings than before.
Code (1)
Tasks
continuous-controlContinuous ControlImitation Learningreinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
VILD: Variational Imitation Learning with Diverse-quality Demonstrations
The goal of imitation learning (IL) is to learn a good policy from high-quality demonstrations. However, the quality of demonstrations in reality can be diverse, since it is easier and cheaper to collect demonstrations f…
continuous-controlContinuous ControlImitation LearningReinforcement LearningRobust Imitation of Diverse Behaviors
Deep generative models have recently shown great promise in imitation learning for motor control. Given enough data, even supervised approaches can do one-shot imitation learning; however, they are vulnerable to cascadin…
Imitation LearningMuJoCoRobust Offline Imitation Learning from Diverse Auxiliary Data
Offline imitation learning enables learning a policy solely from a set of expert demonstrations, without any environment interaction. To alleviate the issue of distribution shift arising due to the small amount of expert…
Imitation LearningTrajectory VAE for multi-modal imitation
We address the problem of imitating multi-modal expert demonstrations in sequential decision making problems. In many practical applications, for example video games, behavioural demonstrations are readily available that…
continuous-controlContinuous ControlDecision MakingImitation Learning+2Imitation from Diverse Behaviors: Wasserstein Quality Diversity Imitation Learning with Single-Step Archive Exploration
Learning diverse and high-performance behaviors from a limited set of demonstrations is a grand challenge. Traditional imitation learning methods usually fail in this task because most of them are designed to learn one s…
continuous-controlContinuous ControlDiversityImitation Learning+1