Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection
The exploration problem is one of the main challenges in deep reinforcement learning (RL). Recent promising works tried to handle the problem with population-based methods, which collect samples with diverse behaviors derived from a population of different exploratory policies. Adaptive policy selection has been adopted for behavior control. However, the behavior selection space is largely limited by the predefined policy population, which further limits behavior diversity. In this paper, we propose a general framework called Learnable Behavioral Control (LBC) to address the limitation, which a) enables a significantly enlarged behavior selection space via formulating a hybrid behavior mapping from all policies; b) constructs a unified learnable process for behavior selection. We introduce LBC into distributed off-policy actor-critic methods and achieve behavior control via optimizing the selection of the behavior mappings with bandit-based meta-controllers. Our agents have achieved 10077.52% mean human normalized score and surpassed 24 human world records within 1B training frames in the Arcade Learning Environment, which demonstrates our significant state-of-the-art (SOTA) performance without degrading the sample efficiency.
Code (0)
등록된 구현이 없습니다.
Tasks
Atari GamesDeep Reinforcement LearningDiversityReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Mask Atari for Deep Reinforcement Learning as POMDP Benchmarks
We present Mask Atari, a new benchmark to help solve partially observable Markov decision process (POMDP) problems with Deep Reinforcement Learning (DRL)-based approaches. To achieve a simulation environment for the POMD…
Atari GamesDeep Reinforcement Learningreinforcement-learningReinforcement Learning+1Estimating Central, Peripheral, and Temporal Visual Contributions to Human Decision Making in Atari Games
We study how different visual information sources contribute to human decision making in dynamic visual environments. Using Atari-HEAD, a large-scale Atari gameplay dataset with synchronized eye-tracking, we introduce a …
Decision MakingAtari GamesHackAtari: Atari Learning Environments for Robust and Continual Reinforcement Learning
Artificial agents' adaptability to novelty and alignment with intended behavior is crucial for their effective deployment. Reinforcement learning (RL) leverages novelty as a means of exploration, yet agents often struggl…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Evolving simple programs for playing Atari games
Cartesian Genetic Programming (CGP) has previously shown capabilities in image processing tasks by evolving programs with a function set specialized for computer vision. A similar approach can be applied to Atari playing…
Atari GamesMastering Atari with Discrete World Models
Intelligent agents need to generalize from past experience to achieve goals in complex environments. World models facilitate such generalization and allow learning behaviors from imagined outcomes to increase sample-effi…
Atari GamesGPU