홈 › Papers

Towards Consistent Performance on Atari using Expert Demonstrations

2019-05-01 · ICLR 2019 5 · Tobias Pohlen, Bilal Piot, Todd Hester, Mohammad Gheshlaghi Azar, Dan Horgan, David Budden, Gabriel Barth-Maron, Hado van Hasselt, John Quan, Mel Večerík, Matteo Hessel, Rémi Munos, Olivier Pietquin

Despite significant advances in the field of deep Reinforcement Learning (RL), today's algorithms still fail to learn human-level policies consistently over a set of diverse tasks such as Atari 2600 games. We identify three key challenges that any algorithm needs to master in order to perform well on all games: processing diverse reward distributions, reasoning over long time horizons, and exploring efficiently. In this paper, we propose an algorithm that addresses each of these challenges and is able to learn human-level policies on nearly all Atari games. A new transformed Bellman operator allows our algorithm to process rewards of varying densities and scales; an auxiliary temporal consistency loss allows us to train stably using a discount factor of 0.999 (instead of 0.99) extending the effective planning horizon by an order of magnitude; and we ease the exploration problem by using human demonstrations that guide the agent towards rewarding states. When tested on a set of 42 Atari games, our algorithm exceeds the performance of an average human on 40 games using a common set of hyper parameters.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Atari GamesDeep Reinforcement LearningReinforcement Learning Reinforcement Learning (RL)

Bayesian Q-learning With Imperfect Expert Demonstrations

2022-10-01 · Fengdi Che, Xiru Zhu, Doina Precup, David Meger 외

Guided exploration with expert demonstrations improves data efficiency for reinforcement learning, but current algorithms often overuse expert information. We propose a novel algorithm to speed up Q-learning with the hel…

Atari Games Q-Learning reinforcement-learning Reinforcement Learning (RL)

Reward learning from human preferences and demonstrations in Atari

2018-11-15 · NeurIPS 2018 12 · Borja Ibarz, Jan Leike, Tobias Pohlen, Geoffrey Irving 외

To solve complex real-world problems with reinforcement learning, we cannot rely on manually specified reward functions. Instead, we can have humans communicate an objective to the agent directly. In this work, we combin…

Atari Games Deep Reinforcement Learning Imitation Learning reinforcement-learning+2

Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration

2025-06-25 · Heyang Zhao, Xingrui Yu, David M. Bossens, Ivor W. Tsang 외

Imitation learning is a central problem in reinforcement learning where the goal is to learn a policy that mimics the expert's behavior. In practice, it is often challenging to learn the expert policy from a limited numb…

Imitation Learning MuJoCo

Pretrain Soft Q-Learning with Imperfect Demonstrations

2019-05-09 · Xiaoqin Zhang, Yunfei Li, Huimin Ma, Xiong Luo

Pretraining reinforcement learning methods with demonstrations has been an important concept in the study of reinforcement learning since a large amount of computing power is spent on online simulations with existing rei…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Observe and Look Further: Achieving Consistent Performance on Atari

2018-05-29 · Tobias Pohlen, Bilal Piot, Todd Hester, Mohammad Gheshlaghi Azar 외

Atari Games Deep Reinforcement Learning Montezuma's Revenge Reinforcement Learning+1