Modularization of End-to-End Learning: Case Study in Arcade Games
Complex environments and tasks pose a difficult problem for holistic end-to-end learning approaches. Decomposition of an environment into interacting controllable and non-controllable objects allows supervised learning for non-controllable objects and universal value function approximator learning for controllable objects. Such decomposition should lead to a shorter learning time and better generalisation capability. Here, we consider arcade-game environments as sets of interacting objects (controllable, non-controllable) and propose a set of functional modules that are specialized on mastering different types of interactions in a broad range of environments. The modules utilize regression, supervised learning, and reinforcement learning algorithms. Results of this case study in different Atari games suggest that human-level performance can be achieved by a learning agent within a human amount of game experience (10-15 minutes game time) when a proper decomposition of an environment or a task is provided. However, automatization of such decomposition remains a challenging problem. This case study shows how a model of a causal structure underlying an environment or a task can benefit learning time and generalization capability of the agent, and argues in favor of exploiting modular structure in contrast to using pure end-to-end learning approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Atari GamesReinforcement LearningSimilar Papers 제목 키워드 기반
Meta Arcade: A Configurable Environment Suite for Meta-Learning
Most approaches to deep reinforcement learning (DRL) attempt to solve a single task at a time. As a result, most existing research benchmarks consist of individual games or suites of games that have common interfaces but…
Deep Reinforcement LearningMeta-LearningTransfer LearningMultiplayer Support for the Arcade Learning Environment
The Arcade Learning Environment ("ALE") is a widely used library in the reinforcement learning community that allows easy programmatic interfacing with Atari 2600 games, via the Stella emulator. We introduce a publicly a…
Atari Gamesreinforcement-learningReinforcement Learning (RL)Atari-5: Distilling the Arcade Learning Environment down to Five Games
The Arcade Learning Environment (ALE) has become an essential benchmark for assessing the performance of reinforcement learning algorithms. However, the computational cost of generating results on the entire 57-game data…
Atari GamesBenchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment
This paper provides an empirical evaluation of recently developed exploration algorithms within the Arcade Learning Environment (ALE). We study the use of different reward bonuses that incentives exploration in reinforce…
Atari GamesBenchmarkingMontezuma's RevengeReinforcement LearningDeep Learning for Video Game Playing
In this article, we review recent Deep Learning advances in the context of how they have been applied to play different types of video games such as first-person shooters, arcade games, and real-time strategy games. We a…
BIG-bench Machine LearningDeep LearningReal-Time Strategy Games