Prioritized Sequence Experience Replay

2019-05-25 · Marc Brittain, Josh Bertram, Xuxi Yang, Peng Wei

Experience replay is widely used in deep reinforcement learning algorithms and allows agents to remember and learn from experiences from the past. In an effort to learn more efficiently, researchers proposed prioritized experience replay (PER) which samples important transitions more frequently. In this paper, we propose Prioritized Sequence Experience Replay (PSER) a framework for prioritizing sequences of experience in an attempt to both learn more efficiently and to obtain better performance. We compare the performance of PER and PSER sampling techniques in a tabular Q-learning environment and in DQN on the Atari 2600 benchmark. We prove theoretically that PSER is guaranteed to converge faster than PER and empirically show PSER substantially improves upon PER.

📄 PDF Abstract BibTeX arXiv:1905.12726

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Prioritized Experience Replay Prioritized Experience Replay is a type of experience replay in reinforcement learning where we more frequently replay…

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Double Prioritized State Recycled Experience Replay

2020-07-08 · Fanchen Bu, Dong Eui Chang

Experience replay enables online reinforcement learning agents to store and reuse the previous experiences of interacting with the environment. In the original method, the experiences are sampled and replayed uniformly a…

Atari Games

A novel DDPG method with prioritized experience replay

2017-10-01 · IEEE International Conference on Systems, Man and Cybernetics (SMC) 2017 10 · Yuenan Hou, Lifeng Liu, Qing Wei, Xudong Xu 외

Recently, a state-of-the-art algorithm, called deep deterministic policy gradient (DDPG), has achieved good performance in many continuous control tasks in the MuJoCo simulator. To further improve the efficiency of the e…

continuous-control Continuous Control MuJoCo OpenAI Gym

Prioritized Experience Replay

2015-11-18 · Tom Schaul, John Quan, Ioannis Antonoglou, David Silver

Experience replay lets online reinforcement learning agents remember and reuse experiences from the past. In prior work, experience transitions were uniformly sampled from a replay memory. However, this approach simply r…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Improving Experience Replay with Successor Representation

2021-11-29 · Yizhi Yuan, Marcelo G Mattar

Prioritized experience replay is a reinforcement learning technique whereby agents speed up learning by replaying useful past experiences. This usefulness is quantified as the expected gain from replaying the experience,…

Atari Games

DQN Performance with Epsilon Greedy Policies and Prioritized Experience Replay

2025-11-05 · Daniel Perkins, Oscar J. Escobar, Luke Green arxiv

We present a detailed study of Deep Q-Networks in finite environments, emphasizing the impact of epsilon-greedy exploration schedules and prioritized experience replay. Through systematic experimentation, we evaluate how…

Reinforcement Learning