홈 › Papers

Reverb: A Framework For Experience Replay

2021-02-09 · Albin Cassirer, Gabriel Barth-Maron, Eugene Brevdo, Sabela Ramos, Toby Boyd, Thibault Sottiaux, Manuel Kroiss

A central component of training in Reinforcement Learning (RL) is Experience: the data used for training. The mechanisms used to generate and consume this data have an important effect on the performance of RL algorithms. In this paper, we introduce Reverb: an efficient, extensible, and easy to use system designed specifically for experience replay in RL. Reverb is designed to work efficiently in distributed configurations with up to thousands of concurrent clients. The flexible API provides users with the tools to easily and accurately configure the replay buffer. It includes strategies for selecting and removing elements from the buffer, as well as options for controlling the ratio between sampled and inserted elements. This paper presents the core design of Reverb, gives examples of how it can be applied, and provides empirical results of Reverb's performance characteristics.

📄 PDF Abstract BibTeX arXiv:2102.04736

Code (1)

deepmind/reverb 공식 구현 tf

Tasks

Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning Models

2023-10-08 · Hanjing Wang, Man-Kit Sit, Congjie He, Ying Wen 외

This paper introduces a distributed, GPU-centric experience replay system, GEAR, designed to perform scalable reinforcement learning (RL) with large sequence models (such as transformers). With such models, existing syst…

GPU Reinforcement Learning (RL)

Prioritized Experience Replay

2015-11-18 · Tom Schaul, John Quan, Ioannis Antonoglou, David Silver

Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory. However, this approach simply r…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Experience Replay Optimization

2019-06-19 · Daochen Zha, Kwei-Herng Lai, Kaixiong Zhou, Xia Hu

Experience replay enables reinforcement learning agents to memorize and reuse past experiences, just as humans replay memories for the situation at hand. Contemporary off-policy algorithms either replay past experiences …

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Deep In-GPU Experience Replay

2018-01-09 · Ben Parr

Experience replay allows a reinforcement learning agent to train on samples from a large amount of the most recent experiences. A simple in-RAM experience replay stores these most recent experiences in a list in RAM, and…

CPU GPU Reinforcement Learning

Prioritized Sequence Experience Replay

2019-05-25 · Marc Brittain, Josh Bertram, Xuxi Yang, Peng Wei

Experience replay is widely used in deep reinforcement learning algorithms and allows agents to remember and learn from experiences from the past. In an effort to learn more efficiently, researchers proposed prioritized …

Deep Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1