On Combining Expert Demonstrations in Imitation Learning via Optimal Transport
Imitation learning (IL) seeks to teach agents specific tasks through expert demonstrations. One of the key approaches to IL is to define a distance between agent and expert and to find an agent policy that minimizes that distance. Optimal transport methods have been widely used in imitation learning as they provide ways to measure meaningful distances between agent and expert trajectories. However, the problem of how to optimally combine multiple expert demonstrations has not been widely studied. The standard method is to simply concatenate state (-action) trajectories, which is problematic when trajectories are multi-modal. We propose an alternative method that uses a multi-marginal optimal transport distance and enables the combination of multiple and diverse state-trajectories in the OT sense, providing a more sensible geometric average of the demonstrations. Our approach enables an agent to learn from several experts, and its efficiency is analyzed on OpenAI Gym control environments and demonstrates that the standard method is not always optimal.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningOpenAI GymSimilar Papers 제목 키워드 기반
Watch and Match: Supercharging Imitation with Regularized Optimal Transport
Imitation learning holds tremendous promise in learning policies efficiently for complex decision making problems. Current state-of-the-art algorithms often use inverse reinforcement learning (IRL), where given a set of …
Decision MakingImitation LearningProvable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behavior
We propose a theoretical framework for studying behavior cloning of complex expert demonstrations using generative modeling. Our framework invokes low-level controllers - either learned or implicit in position-command co…
Data AugmentationWasserstein Adversarial Imitation Learning
Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they u…
Imitation Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Score-Based Diffusion Policy Compatible with Reinforcement Learning via Optimal Transport
Diffusion policies have shown promise in learning complex behaviors from demonstrations, particularly for tasks requiring precise control and long-term planning. However, they face challenges in robustness when encounter…
Imitation LearningCross-Domain Imitation Learning via Optimal Transport
Cross-domain imitation learning studies how to leverage expert demonstrations of one agent to train an imitation agent with a different embodiment or morphology. Comparing trajectories and stationary distributions betwee…
continuous-controlContinuous ControlImitation Learning