$f$-GAIL: Learning $f$-Divergence for Generative Adversarial Imitation Learning
Imitation learning (IL) aims to learn a policy from expert demonstrations that minimizes the discrepancy between the learner and expert behaviors. Various imitation learning algorithms have been proposed with different pre-determined divergences to quantify the discrepancy. This naturally gives rise to the following question: Given a set of expert demonstrations, which divergence can recover the expert policy more accurately with higher data efficiency? In this work, we propose $f$-GAIL, a new generative adversarial imitation learning (GAIL) model, that automatically learns a discrepancy measure from the $f$-divergence family as well as a policy capable of producing expert-like behaviors. Compared with IL baselines with various predefined divergence measures, $f$-GAIL learns better policies with higher data efficiency in six physics-based control tasks.
Code (1)
Tasks
Imitation LearningSimilar Papers 제목 키워드 기반
f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
Imitation learning (IL) aims to learn a policy from expert demonstrations that minimizes the discrepancy between the learner and expert behaviors. Various imitation learning algorithms have been proposed with different p…
Imitation LearningRisk-Sensitive Generative Adversarial Imitation Learning
We study risk-sensitive imitation learning where the agent's goal is to perform at least as well as the expert in terms of a risk profile. We first formulate our risk-sensitive imitation learning setting. We consider the…
Imitation LearningMuJoCoExploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
Generative Adversarial Imitation Learning (GAIL) stands as a cornerstone approach in imitation learning. This paper investigates the gradient explosion in two types of GAIL: GAIL with deterministic policy (DE-GAIL) and G…
Imitation LearningWasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration
The generative adversarial imitation learning (GAIL) has provided an adversarial learning framework for imitating expert policy from demonstrations in high-dimensional continuous tasks. However, almost all GAIL and its e…
continuous-controlContinuous ControlImitation LearningMuJoCoC-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory
Generative Adversarial Imitation Learning (GAIL) trains a generative policy to mimic a demonstrator. It uses on-policy Reinforcement Learning (RL) to optimize a reward signal derived from a GAN-like discriminator. A majo…
Imitation LearningMuJoCoReinforcement Learning (RL)