Recurrent Experience Replay in Distributed Reinforcement Learning

2019-05-01 · ICLR 2019 5 · Steven Kapturowski, Georg Ostrovski, Will Dabney, John Quan, Remi Munos

Building on the recent successes of distributed training of RL agents, in this paper we investigate the training of RNN-based RL agents from distributed prioritized experience replay. We study the effects of parameter lag resulting in representational drift and recurrent state staleness and empirically derive an improved training strategy. Using a single network architecture and fixed set of hyperparameters, the resulting agent, Recurrent Replay Distributed DQN, quadruples the previous state of the art on Atari-57, and surpasses the state of the art on DMLab-30. It is the first agent to exceed human-level performance in 52 of the 57 Atari games.

📄 PDF Abstract BibTeX

Code (3)

garymm/earl/tree/master/earl/agents/r2d2 jax

michaelnny/deep_rl_zoo pytorch

opendilab/DI-engine/blob/main/ding/policy/r2d2.py pytorch

Tasks

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

R2D2 Building on the recent successes of distributed training of RL agents, R2D2 is an RL approach that trains a RNN-based RL agents from distributed prioritized experience replay.…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Accelerating Distributed Deep Reinforcement Learning by In-Network Experience Sampling

2021-10-26 · Masaki Furukawa, Hiroki Matsutani

A computing cluster that interconnects multiple compute nodes is used to accelerate distributed reinforcement learning based on DQN (Deep Q-Network). In distributed reinforcement learning, Actor nodes acquire experiences…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Distributed Prioritized Experience Replay

2018-03-02 · ICLR 2018 1 · Dan Horgan, John Quan, David Budden, Gabriel Barth-Maron 외

We propose a distributed architecture for deep reinforcement learning at scale, that enables agents to learn effectively from orders of magnitude more data than previously possible. The algorithm decouples acting from le…

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Proxy Experience Replay: Federated Distillation for Distributed Reinforcement Learning

2020-05-13 · Han Cha, Jihong Park, Hyesung Kim, Mehdi Bennis 외

Traditional distributed deep reinforcement learning (RL) commonly relies on exchanging the experience replay memory (RM) of each agent. Since the RM contains all state observations and action policy history, it may incur…

Clustering Data Augmentation Deep Reinforcement Learning Privacy Preserving+3

Distributed Soft Actor-Critic with Multivariate Reward Representation and Knowledge Distillation

2019-11-29 · Dmitry Akimov

In this paper, we describe NeurIPS 2019 Learning to Move - Walk Around challenge physics-based environment and present our solution to this competition which scored 1303.727 mean reward points and took 3rd place. Our met…

Knowledge Distillation reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Quantum deep Q learning with distributed prioritized experience replay

2023-04-19 · Samuel Yen-Chi Chen

This paper introduces the QDQN-DPER framework to enhance the efficiency of quantum reinforcement learning (QRL) in solving sequential decision tasks. The framework incorporates prioritized experience replay and asynchron…

Q-Learning reinforcement-learning Reinforcement Learning