홈 › Papers

Massively Parallel Methods for Deep Reinforcement Learning

2015-07-15 · Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, Shane Legg, Volodymyr Mnih, Koray Kavukcuoglu, David Silver

We present the first massively distributed architecture for deep reinforcement learning. This architecture uses four main components: parallel actors that generate new behaviour; parallel learners that are trained from stored experience; a distributed neural network to represent the value function or behaviour policy; and a distributed store of experience. We used our architecture to implement the Deep Q-Network algorithm (DQN). Our distributed algorithm was applied to 49 games from Atari 2600 games from the Arcade Learning Environment, using identical hyperparameters. Our performance surpassed non-distributed DQN in 41 of the 49 games and also reduced the wall-time required to achieve these results by an order of magnitude on most games.

📄 PDF Abstract BibTeX arXiv:1507.04296

Code (3)

londoed/Gorila tf

londoed/Kortex tf

nandomp/AICollaboratory

Tasks

Atari GamesDeep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Trust-Region Diffusion Policies for Massively Parallel On-Policy RL

2026-06-13 · Huy Le, Onur Celik, Denis Blessing, Tai Hoang 외 arxiv

Reinforcement learning with massively parallel simulations has become a standard framework for developing robust, deployable policies; however, most existing approaches still rely on simple Gaussian policy parameterizati…

Reinforcement Learning

Benchmarking Massively Parallelized Multi-Task Reinforcement Learning for Robotics Tasks

2025-07-31 · Viraj Joshi, Zifan Xu, Bo Liu, Peter Stone 외 arxiv

Multi-task Reinforcement Learning (MTRL) has emerged as a critical training paradigm for applying reinforcement learning (RL) to a set of complex real-world robotic tasks, which demands a generalizable and robust policy.…

Reinforcement Learning

Parallel $Q$-Learning: Scaling Off-policy Reinforcement Learning under Massively Parallel Simulation

2023-07-24 · Zechu Li, Tao Chen, Zhang-Wei Hong, Anurag Ajay 외

Reinforcement learning is time-consuming for complex tasks due to the need for large amounts of training data. Recent advances in GPU-based simulation, such as Isaac Gym, have sped up data collection thousands of times o…

GPU Q-Learning reinforcement-learning

Practical Massively Parallel Monte-Carlo Tree Search Applied to Molecular Design

2020-06-18 · ICLR 2021 1 · Xiufeng Yang, Tanuj Kr Aasawat, Kazuki Yoshizoe

It is common practice to use large computational resources to train neural networks, as is known from many examples, such as reinforcement learning applications. However, while massively parallel computing is often used …

Combinatorial Optimization CPU

Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning

2021-09-24 · Nikita Rudin, David Hoeller, Philipp Reist, Marco Hutter

In this work, we present and study a training set-up that achieves fast policy generation for real-world robotic tasks by using massive parallelism on a single workstation GPU. We analyze and discuss the impact of differ…

Deep Reinforcement Learning GPU reinforcement-learning Reinforcement Learning (RL)