Paused Agent Replay Refresh

2022-09-26 · Benjamin Parr

Reinforcement learning algorithms have become more complex since the invention of target networks. Unfortunately, target networks have not kept up with this increased complexity, instead requiring approximate solutions to be computationally feasible. These approximations increase noise in the Q-value targets and in the replay sampling distribution. Paused Agent Replay Refresh (PARR) is a drop-in replacement for target networks that supports more complex learning algorithms without this need for approximation. Using a basic Q-network architecture, and refreshing the novelty values, target values, and replay sampling distribution, PARR gets 2500 points in Montezuma's Revenge after only 30.9 million Atari frames. Finally, interpreting PARR in the context of carbon-based learning offers a new reason for sleep.

📄 PDF Abstract BibTeX arXiv:2209.13398

Code (0)

등록된 구현이 없습니다.

Tasks

Montezuma's Revenge reinforcement-learning Reinforcement Learning (RL)

Lucid Dreaming for Experience Replay: Refreshing Past States with the Current Policy

2020-09-29 · Yunshu Du, Garrett Warnell, Assefaw Gebremedhin, Peter Stone 외

Experience replay (ER) improves the data efficiency of off-policy reinforcement learning (RL) algorithms by allowing an agent to store and reuse its past experiences in a replay buffer. While many techniques have been pr…

Atari Games Reinforcement Learning (RL)

Uniform Herding: Exemplar Replay with Representation Refresh

2026-08-13 · Krishna Subedi arxiv

As the feature representation changes, replay must preserve the earlier classes. However, only a bounded active exemplar set can be replayed. We propose Uniform Herding, which allocates the current active set across obse…

SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain

2026-05-18 · Lingtao Mao, Huangyu Dai, Xinyu Sun, Zihan Liang 외 arxiv

Multimodal large language models are increasingly used as agent backbones that understand multimodal inputs, plan retrieval actions, invoke external tools, and reason over retrieved information. Yet existing benchmarks r…

Visual Grounding

Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Training

2026-02-28 · Xi Wang, Wenbo Lu, Shengjie Wang arxiv

Generative Flow Networks (GFlowNets) enable fine-tuning large language models to approximate reward-proportional posteriors, but they remain prone to mode collapse, manifesting as prefix collapse and length bias. We attr…

SentinelBench: A Benchmark for Long-Running Monitoring Agents

2026-06-03 · Matheus Kunzler Maldaner, Adam Fourney, Amanda Swearngin, Hussein Mozannar 외 arxiv

AI agents are increasingly asked to carry out work that spans minutes, hours, or longer. Yet the default model of agent behavior is continuous action: issuing tool calls, refreshing pages, searching for alternatives, or …