Provably Efficient Imitation Learning from Observation Alone

2019-05-27 · Wen Sun, Anirudh Vemula, Byron Boots, J. Andrew Bagnell

We study Imitation Learning (IL) from Observations alone (ILFO) in large-scale MDPs. While most IL algorithms rely on an expert to directly provide actions to the learner, in this setting the expert only supplies sequences of observations. We design a new model-free algorithm for ILFO, Forward Adversarial Imitation Learning (FAIL), which learns a sequence of time-dependent policies by minimizing an Integral Probability Metric between the observation distributions of the expert policy and the learner. FAIL is the first provably efficient algorithm in ILFO setting, which learns a near-optimal policy with a number of samples that is polynomial in all relevant parameters but independent of the number of unique observations. The resulting theory extends the domain of provably sample efficient learning algorithms beyond existing results, which typically only consider tabular reinforcement learning settings or settings that require access to a near-optimal reset distribution. We also investigate the extension of FAIL in a model-based setting. Finally we demonstrate the efficacy of FAIL on multiple OpenAI Gym control tasks.

📄 PDF Abstract BibTeX arXiv:1905.10948

Code (1)

wensun/Imitation-Learning-from-Observation 공식 구현 tf

Tasks

Imitation LearningOpenAI Gym Reinforcement Learning

MobILE: Model-Based Imitation Learning From Observation Alone

2021-02-22 · NeurIPS 2021 12 · Rahul Kidambi, Jonathan Chang, Wen Sun

This paper studies Imitation Learning from Observations alone (ILFO) where the learner is presented with expert demonstrations that consist only of states visited by an expert (without access to actions taken by the expe…

Imitation Learning model OpenAI Gym

Optimism is All You Need: Model-Based Imitation Learning From Observation Alone

2021-03-09 · ICLR Workshop SSL-RL 2021 5 · Rahul Kidambi, Jonathan Daniel Chang, Wen Sun

This paper studies Imitation Learning from Observations alone (ILFO) where the learner is presented with expert demonstrations that only consist of states encountered by an expert (without access to actions taken by the …

All Imitation Learning OpenAI Gym

Provably Efficient Third-Person Imitation from Offline Observation

2020-02-27 · Aaron Zweig, Joan Bruna

Domain adaptation in imitation learning represents an essential step towards improving generalizability. However, even in the restricted setting of third-person imitation where transfer is between isomorphic Markov Decis…

Domain Adaptation Imitation Learning

On the Limits of Prompt-Conditioned Language Models as General-Purpose Learners

2026-06-22 · David Mguni, Julian Ma, Jun Wang arxiv

Large Language Models (LLMs) are frequently portrayed as general-purpose solvers capable of solving arbitrary tasks. We argue that this view overlooks a fundamental constraint: language is a compressed and capacity-limit…

Hybrid Reinforcement Learning from Offline Observation Alone

2024-06-11 · Yuda Song, J. Andrew Bagnell, Aarti Singh

We consider the hybrid reinforcement learning setting where the agent has access to both offline data and online interactive access. While Reinforcement Learning (RL) research typically assumes offline data contains comp…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)