Fast Policy Learning through Imitation and Reinforcement

2018-05-26 · Ching-An Cheng, Xinyan Yan, Nolan Wagener, Byron Boots

Imitation learning (IL) consists of a set of tools that leverage expert demonstrations to quickly learn policies. However, if the expert is suboptimal, IL can yield policies with inferior performance compared to reinforcement learning (RL). In this paper, we aim to provide an algorithm that combines the best aspects of RL and IL. We accomplish this by formulating several popular RL and IL algorithms in a common mirror descent framework, showing that these algorithms can be viewed as a variation on a single approach. We then propose LOKI, a strategy for policy learning that first performs a small but random number of IL iterations before switching to a policy gradient RL method. We show that if the switching time is properly randomized, LOKI can learn to outperform a suboptimal expert and converge faster than running policy gradient from scratch. Finally, we evaluate the performance of LOKI experimentally in several simulated environments.

📄 PDF Abstract BibTeX arXiv:1805.10413

Code (0)

등록된 구현이 없습니다.

Tasks

Imitation LearningReinforcement Learning Reinforcement Learning (RL)

Learning to Reinforcement Learn by Imitation

2019-05-01 · ICLR 2019 5 · Rosen Kralev, Russell Mendonca, Alvin Zhang, Tianhe Yu 외

Meta-reinforcement learning aims to learn fast reinforcement learning (RL) procedures that can be applied to new tasks or environments. While learning fast RL procedures holds promise for allowing agents to autonomously …

Meta-Learning Meta Reinforcement Learning reinforcement-learning Reinforcement Learning+1

FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control

2026-04-06 · Donghu Kim, Youngdo Lee, Minho Park, Kinam Kim 외 arxiv

Reinforcement learning (RL) is a core approach for robot control when expert demonstrations are unavailable. On-policy methods such as Proximal Policy Optimization (PPO) are widely used for their stability, but their rel…

Reinforcement Learning

Policy Optimization by Local Improvement through Search

2019-09-25 · Jialin Song, Joe Wenjie Jiang, Amir Yazdanbakhsh, Ebrahim Songhori 외

Imitation learning has emerged as a powerful strategy for learning initial policies that can be refined with reinforcement learning techniques. Most strategies in imitation learning, however, rely on per-step supervision…

Imitation Learning reinforcement-learning Reinforcement Learning (RL)

Temporal Self-Imitation Learning

2026-06-18 · Yinsen Jia, Boyuan Chen arxiv

Long-horizon robot manipulation policies trained with reward shaping can still achieve high return through inefficient interactions, while rare efficient behaviors discovered during training may be forgotten. We argue th…

Reinforcement Learning Robot Manipulation

Accelerating Training in Pommerman with Imitation and Reinforcement Learning

2019-11-12 · Hardik Meisheri, Omkar Shelke, Richa Verma, Harshad Khadilkar

The Pommerman simulation was recently developed to mimic the classic Japanese game Bomberman, and focuses on competitive gameplay in a multi-agent setting. We focus on the 2$\times$2 team version of Pommerman, developed …

Imitation Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)