Error Bounds of Imitating Policies and Environments

2020-10-22 · NeurIPS 2020 12 · Tian Xu, Ziniu Li, Yang Yu

Imitation learning trains a policy by mimicking expert demonstrations. Various imitation methods were proposed and empirically evaluated, meanwhile, their theoretical understanding needs further studies. In this paper, we firstly analyze the value gap between the expert policy and imitated policies by two imitation methods, behavioral cloning and generative adversarial imitation. The results support that generative adversarial imitation can reduce the compounding errors compared to behavioral cloning, and thus has a better sample complexity. Noticed that by considering the environment transition model as a dual agent, imitation learning can also be used to learn the environment model. Therefore, based on the bounds of imitating policies, we further analyze the performance of imitating environments. The results show that environment models can be more effectively imitated by generative adversarial imitation than behavioral cloning, suggesting a novel application of adversarial imitation for model-based reinforcement learning. We hope these results could inspire future advances in imitation learning and model-based reinforcement learning.

📄 PDF Abstract BibTeX arXiv:2010.11876

Code (0)

등록된 구현이 없습니다.

Tasks

Imitation LearningModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Imitating Latent Policies from Observation

2018-05-21 · Ashley D. Edwards, Himanshu Sahni, Yannick Schroecker, Charles L. Isbell

In this paper, we describe a novel approach to imitation learning that infers latent policies directly from state observations. We introduce a method that characterizes the causal effects of latent actions on observation…

Imitation Learning

Synthesizing Programmatic Policies that Inductively Generalize

2020-05-01 · ICLR 2020 1 · Jeevana Priya Inala, Osbert Bastani, Zenna Tavares, Armando Solar-Lezama

Deep reinforcement learning has successfully solved a number of challenging control tasks. However, learned policies typically have difficulty generalizing to novel environments. We propose an algorithm for learning prog…

Deep Reinforcement Learning Imitation Learning Reinforcement Learning (RL)

Robust Asymmetric Learning in POMDPs

2020-12-31 · Andrew Warrington, J. Wilder Lavington, Adam Ścibior, Mark Schmidt 외

Policies for partially observed Markov decision processes can be efficiently learned by imitating policies for the corresponding fully observed Markov decision processes. Unfortunately, existing approaches for this kind …

Imitation Learning

Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning

2022-04-07 · Carl Qi, Pieter Abbeel, Aditya Grover

The goal of imitation learning is to mimic expert behavior from demonstrations, without access to an explicit reward signal. A popular class of approach infers the (unknown) reward function via inverse reinforcement lear…

Imitation Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Imitating Task and Motion Planning with Visuomotor Transformers

2023-05-25 · Murtaza Dalal, Ajay Mandlekar, Caelan Garrett, Ankur Handa 외

Imitation learning is a powerful tool for training robot manipulation policies, allowing them to learn from expert demonstrations without manual programming or trial-and-error. However, common methods of data collection,…

Imitation Learning Motion Planning Robot Manipulation Task and Motion Planning