Imitation Learning from Observation through Optimal Transport
Imitation Learning from Observation (ILfO) is a setting in which a learner tries to imitate the behavior of an expert, using only observational data and without the direct guidance of demonstrated actions. In this paper, we re-examine optimal transport for IL, in which a reward is generated based on the Wasserstein distance between the state trajectories of the learner and expert. We show that existing methods can be simplified to generate a reward function without requiring learned models or adversarial learning. Unlike many other state-of-the-art methods, our approach can be integrated with any RL algorithm and is amenable to ILfO. We demonstrate the effectiveness of this simple approach on a variety of continuous control tasks and find that it surpasses the state of the art in the IlfO setting, achieving expert-level performance across a range of evaluation domains even when observing only a single expert trajectory without actions.
Code (0)
등록된 구현이 없습니다.
Tasks
continuous-controlContinuous ControlImitation LearningSimilar Papers 제목 키워드 기반
Wasserstein Adversarial Imitation Learning
Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they u…
Imitation Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)An Optimal Transport Formulation of Bayes' Law for Nonlinear Filtering Algorithms
This paper presents a variational representation of the Bayes' law using optimal transportation theory. The variational representation is in terms of the optimal transportation between the joint distribution of the (stat…
On Combining Expert Demonstrations in Imitation Learning via Optimal Transport
Imitation learning (IL) seeks to teach agents specific tasks through expert demonstrations. One of the key approaches to IL is to define a distance between agent and expert and to find an agent policy that minimizes that…
Imitation LearningOpenAI GymSemidiscrete optimal transport with unknown costs
Semidiscrete optimal transport is a challenging generalization of the classical transportation problem in linear programming. The goal is to design a joint distribution for two random variables (one continuous, one discr…
Align Your Intents: Offline Imitation Learning via Optimal Transport
Offline Reinforcement Learning (RL) addresses the problem of sequential decision-making by learning optimal policy through pre-collected data, without interacting with the environment. As yet, it has remained somewhat im…
D4RLDecision MakingImitation LearningOffline RL+2