paper-with-me

홈 › Papers

On Bonus Based Exploration Methods In The Arcade Learning Environment

2020-01-01 · ICLR 2020 1 · Adrien Ali Taiga, William Fedus, Marlos C. Machado, Aaron Courville, Marc G. Bellemare

Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration problems such as Montezuma's Revenge (Bellemare et al., 2016). Recently, bonus-based exploration methods, which explore by augmenting the environment reward, have reached above-human average performance on such domains. In this paper we reassess popular bonus-based exploration methods within a common evaluation framework. We combine Rainbow (Hessel et al., 2018) with different exploration bonuses and evaluate its performance on Montezuma's Revenge, Bellemare et al.'s set of hard of exploration games with sparse rewards, and the whole Atari 2600 suite. We find that while exploration bonuses lead to higher score on Montezuma's Revenge they do not provide meaningful gains over the simpler epsilon-greedy scheme. In fact, we find that methods that perform best on that game often underperform epsilon-greedy on easy exploration Atari 2600 games. We find that our conclusions remain valid even when hyperparameters are tuned for these easy-exploration games. Finally, we find that none of the methods surveyed benefit from additional training samples (1 billion frames, versus Rainbow's 200 million) on Bellemare et al.'s hard exploration games. Our results suggest that recent gains in Montezuma's Revenge may be better attributed to architecture change, rather than better exploration schemes; and that the real pace of progress in exploration research for Atari 2600 games may have been obfuscated by good results on a single domain.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Atari GamesMontezuma's RevengeReinforcement Learning

Similar Papers 제목 키워드 기반

Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment

2019-08-06 · Adrien Ali Taïga, William Fedus, Marlos C. Machado, Aaron Courville 외

This paper provides an empirical evaluation of recently developed exploration algorithms within the Arcade Learning Environment (ALE). We study the use of different reward bonuses that incentives exploration in reinforce…

Atari GamesBenchmarkingMontezuma's RevengeReinforcement Learning

On Bonus-Based Exploration Methods in the Arcade Learning Environment

2021-09-22 · Adrien Ali Taïga, William Fedus, Marlos C. Machado, Aaron Courville 외

Research on exploration in reinforcement learning, as applied to Atari 2600 game-playing, has emphasized tackling difficult exploration problems such as Montezuma's Revenge (Bellemare et al., 2016). Recently, bonus-based…

Atari GamesMontezuma's Revenge

Perturbation-based exploration methods in deep reinforcement learning

2020-11-10 · Sneha Aenugu

Recent research on structured exploration placed emphasis on identifying novel states in the state space and incentivizing the agent to revisit them through intrinsic reward bonuses. In this study, we question whether th…

Atari GamesBenchmarkingDeep Reinforcement Learningreinforcement-learning+2

Efficient Potential-based Exploration in Reinforcement Learning using Inverse Dynamic Bisimulation Metric

2023-09-21 · NeurIPS 2023 11

Reward shaping is an effective technique for integrating domain knowledge into reinforcement learning (RL). However, traditional approaches like potential-based reward shaping totally rely on manually designing shaping r…

Exploring Unknown States with Action Balance

2020-03-10 · Yan Song, Yingfeng Chen, Yujing Hu, Changjie Fan

Exploration is a key problem in reinforcement learning. Recently bonus-based methods have achieved considerable successes in environments where exploration is difficult such as Montezuma's Revenge, which assign additiona…

Atari GamesDeep Reinforcement LearningMontezuma's Revengereinforcement-learning+2