Measuring Progress in Deep Reinforcement Learning Sample Efficiency

2021-02-09 · Florian E. Dorner

Sampled environment transitions are a critical input to deep reinforcement learning (DRL) algorithms. Current DRL benchmarks often allow for the cheap and easy generation of large amounts of samples such that perceived progress in DRL does not necessarily correspond to improved sample efficiency. As simulating real world processes is often prohibitively hard and collecting real world experience is costly, sample efficiency is an important indicator for economically relevant applications of DRL. We investigate progress in sample efficiency on Atari games and continuous control tasks by comparing the number of samples that a variety of algorithms need to reach a given performance level according to training curves in the corresponding publications. We find exponential progress in sample efficiency with estimated doubling times of around 10 to 18 months on Atari, 5 to 24 months on state-based continuous control and of around 4 to 9 months on pixel-based continuous control depending on the specific task and performance level.

📄 PDF Abstract BibTeX arXiv:2102.04881

Code (0)

등록된 구현이 없습니다.

Tasks

Atari Games continuous-control Continuous ControlDeep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmark

2021-03-29 · Sharada Mohanty, Jyotish Poonganam, Adrien Gaidon, Andrey Kolobov 외

The NeurIPS 2020 Procgen Competition was designed as a centralized benchmark with clearly defined tasks for measuring Sample Efficiency and Generalization in Reinforcement Learning. Generalization remains one of the most…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Measuring Learning Progress via Gradient-Momentum Coupling

2026-05-07 · Samuel Blad, Martin Längkvist, Amy Loutfi arxiv

Measuring learning progress is essential for curiosity-driven exploration in reinforcement learning, but widely used signals such as prediction error often fail to distinguish meaningful, learnable patterns from random n…

Reinforcement Learning

Towards Revealing the Effectiveness of Small-Scale Fine-tuning in R1-style Reinforcement Learning

2025-05-23 · Yutong Chen, Jiandong Gao, Ji Wu

R1-style Reinforcement Learning (RL) significantly enhances Large Language Models' reasoning capabilities, yet the mechanism behind rule-based RL remains unclear. We found that small-scale SFT has significant influence o…

Math Reinforcement Learning (RL)

State-based Episodic Memory for Multi-Agent Reinforcement Learning

2021-10-19 · Xiao Ma, Wu-Jun Li

Multi-agent reinforcement learning (MARL) algorithms have made promising progress in recent years by leveraging the centralized training and decentralized execution (CTDE) paradigm. However, existing MARL algorithms stil…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+3

Active Measuring in Reinforcement Learning With Delayed Negative Effects

2025-10-16 · Daiqi Gao, Ziping Xu, Aseel Rawashdeh, Predrag Klasnja 외 arxiv

Measuring states in reinforcement learning (RL) can be costly in real-world settings and may negatively influence future outcomes. We introduce the Actively Observable Markov Decision Process (AOMDP), where an agent not …