Learning from Imperfect Demonstrations via Adversarial Confidence Transfer
Existing learning from demonstration algorithms usually assume access to expert demonstrations. However, this assumption is limiting in many real-world applications since the collected demonstrations may be suboptimal or even consist of failure cases. We therefore study the problem of learning from imperfect demonstrations by learning a confidence predictor. Specifically, we rely on demonstrations along with their confidence values from a different correspondent environment (source environment) to learn a confidence predictor for the environment we aim to learn a policy in (target environment -- where we only have unlabeled demonstrations.) We learn a common latent space through adversarial distribution matching of multi-length partial trajectories to enable the transfer of confidence across source and target environments. The learned confidence reweights the demonstrations to enable learning more from informative demonstrations and discarding the irrelevant ones. Our experiments in three simulated environments and a real robot reaching task demonstrate that our approach learns a policy with the highest expected return.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Imitation Learning from Imperfect Demonstration
Imitation learning (IL) aims to learn an optimal policy from demonstrations. However, such demonstrations are often imperfect since collecting optimal ones is costly. To effectively learn from imperfect demonstrations, w…
Imitation LearningUnlabeled Imperfect Demonstrations in Adversarial Imitation Learning
Adversarial imitation learning has become a widely used imitation learning framework. The discriminator is often trained by taking expert demonstrations and policy trajectories as examples respectively from two categorie…
Imitation LearningMuJoCoDemoDICE: Offline Imitation Learning with Supplementary Imperfect Demonstrations
We consider offline imitation learning (IL), which aims to mimic the expert's behavior from its demonstration without further interaction with the environment. One of the main challenges in offline IL is to deal with th…
Imitation LearningAn Imitation from Observation Approach to Transfer Learning with Dynamics Mismatch
We examine the problem of transferring a policy learned in a source environment to a target environment with different dynamics, particularly in the case where it is critical to reduce the amount of interaction with the …
Transfer LearningAdaptive t-Momentum-based Optimization for Unknown Ratio of Outliers in Amateur Data in Imitation Learning
Behavioral cloning (BC) bears a high potential for safe and direct transfer of human skills to robots. However, demonstrations performed by human operators often contain noise or imperfect behaviors that can affect the e…
Imitation Learning