Deterministic Policy Imitation Gradient Algorithm

2018-01-01 · ICLR 2018 1 · Fumihiro Sasaki, Atsuo Kawaguchi

The goal of imitation learning (IL) is to enable a learner to imitate an expert’s behavior given the expert’s demonstrations. Recently, generative adversarial imitation learning (GAIL) has successfully achieved it even on complex continuous control tasks. However, GAIL requires a huge number of interactions with environment during training. We believe that IL algorithm could be more applicable to the real-world environments if the number of interactions could be reduced. To this end, we propose a model free, off-policy IL algorithm for continuous control. The keys of our algorithm are two folds: 1) adopting deterministic policy that allows us to derive a novel type of policy gradient which we call deterministic policy imitation gradient (DPIG), 2) introducing a function which we call state screening function (SSF) to avoid noisy policy updates with states that are not typical of those appeared on the expert’s demonstrations. Experimental results show that our algorithm can achieve the goal of IL with at least tens of times less interactions than GAIL on a variety of continuous control tasks.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous ControlImitation Learning

Deterministic Policy Gradient Algorithms

2014-06-22 · International Conference on Machine Learning 2014 6 · David Silver, Guy Lever, Nicolas Heess, Thomas Degris 외

In this paper we consider deterministic policy gradient algorithms for reinforcement learning with continuous actions. The deterministic policy gradient has a particularly appealing form: it is the expected gradient of t…

Form reinforcement-learning Reinforcement Learning (RL)

Learning Neural Parsers with Deterministic Differentiable Imitation Learning

2018-06-20 · Tanmay Shankar, Nicholas Rhinehart, Katharina Muelling, Kris M. Kitani

We explore the problem of learning to decompose spatial tasks into segments, as exemplified by the problem of a painting robot covering a large object. Inspired by the ability of classical decision tree algorithms to con…

Imitation Learning Reinforcement Learning

Deterministic Value-Policy Gradients

2019-09-09 · Qingpeng Cai, Ling Pan, Pingzhong Tang

Reinforcement learning algorithms such as the deep deterministic policy gradient algorithm (DDPG) has been widely used in continuous control tasks. However, the model-free DDPG algorithm suffers from high sample complexi…

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+2

Deterministic Policy Gradients With General State Transitions

2018-07-10 · Qingpeng Cai, Ling Pan, Pingzhong Tang

We study a reinforcement learning setting, where the state transition function is a convex combination of a stochastic continuous function and a deterministic function. Such a setting generalizes the widely-studied stoch…

continuous-control Continuous Control Reinforcement Learning

Softmax Deep Double Deterministic Policy Gradients

2020-10-19 · NeurIPS 2020 12 · Ling Pan, Qingpeng Cai, Longbo Huang

A widely-used actor-critic reinforcement learning algorithm for continuous control, Deep Deterministic Policy Gradients (DDPG), suffers from the overestimation problem, which can negatively affect the performance. Althou…

continuous-control Continuous Control