Action-Conditional Video Prediction using Deep Networks in Atari Games
Motivated by vision-based reinforcement learning (RL) problems, in particular Atari games from the recent benchmark Aracade Learning Environment (ALE), we consider spatio-temporal prediction problems where future (image-)frames are dependent on control variables or actions as well as previous frames. While not composed of natural scenes, frames in Atari games are high-dimensional in size, can involve tens of objects with one or more objects being controlled by the actions directly and many other objects being influenced indirectly, can involve entry and departure of objects, and can involve deep partial observability. We propose and evaluate two deep neural network architectures that consist of encoding, action-conditional transformation, and decoding layers based on convolutional neural networks and recurrent neural networks. Experimental results show that the proposed architectures are able to generate visually-realistic frames that are also useful for control over approximately 100-step action-conditional futures in some games. To the best of our knowledge, this paper is the first to make and evaluate long-term predictions on high-dimensional video conditioned by control inputs.
Code (1)
Tasks
Atari GamesReinforcement LearningReinforcement Learning (RL)Video PredictionSimilar Papers 제목 키워드 기반
Model-Based Reinforcement Learning for Atari
Model-free reinforcement learning (RL) can be used to learn effective policies for complex tasks, such as Atari games, even from image observations. However, this typically requires very large amounts of interaction -- s…
Atari GamesAtari Games 100kmodelModel-based Reinforcement Learning+4Model Based Reinforcement Learning for Atari
Model-free reinforcement learning (RL) can be used to learn effective policies for complex tasks, such as Atari games, even from image observations. However, this typically requires very large amounts of interaction -- s…
Atari GamesmodelModel-based Reinforcement Learningreinforcement-learning+3A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games
Reinforcement learning is concerned with identifying reward-maximizing behaviour policies in environments that are initially unknown. State-of-the-art reinforcement learning approaches, such as deep Q-networks, are model…
Atari GamesModel-based Reinforcement LearningPredictionreinforcement-learning+2Disentangling Controllable Object through Video Prediction Improves Visual Reinforcement Learning
In many vision-based reinforcement learning (RL) problems, the agent controls a movable object in its visual field, e.g., the player's avatar in video games and the robotic arm in visual grasping and manipulation. Levera…
Atari GamesObjectPredictionreinforcement-learning+3Estimates for the Branching Factors of Atari Games
The branching factor of a game is the average number of new states reachable from a given state. It is a widely used metric in AI research on board games, but less often computed or discussed for videogames. This paper p…
Atari GamesBoard Games