Revisiting the Minimalist Approach to Offline Reinforcement Learning

2023-05-16 · NeurIPS 2023 11 · Denis Tarasov, Vladislav Kurenkov, Alexander Nikulin, Sergey Kolesnikov

Recent years have witnessed significant advancements in offline reinforcement learning (RL), resulting in the development of numerous algorithms with varying degrees of complexity. While these algorithms have led to noteworthy improvements, many incorporate seemingly minor design choices that impact their effectiveness beyond core algorithmic advances. However, the effect of these design choices on established baselines remains understudied. In this work, we aim to bridge this gap by conducting a retrospective analysis of recent works in offline RL and propose ReBRAC, a minimalistic algorithm that integrates such design elements built on top of the TD3+BC method. We evaluate ReBRAC on 51 datasets with both proprioceptive and visual state spaces using D4RL and V-D4RL benchmarks, demonstrating its state-of-the-art performance among ensemble-free methods in both offline and offline-to-online settings. To further illustrate the efficacy of these design choices, we perform a large-scale ablation study and hyperparameter sensitivity analysis on the scale of thousands of experiments.

📄 PDF Abstract BibTeX arXiv:2305.09836

Code (3)

dt6a/rebrac 공식 구현 jax

adamjelley/efficientofflinerl pytorch

takuseno/d3rlpy pytorch

Tasks

D4RL Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Sim2O: Efficient Offline-to-Online MARL via Joint Action Composition

2026-06-19 · Bingchang Song, Yiqin Yang arxiv

Offline-to-online adaptation serves as a pivotal paradigm for mitigating the prohibitive cost of online exploration by bootstrapping reinforcement learning from offline datasets. While this paradigm has been extensively …

Multi-agent Reinforcement Learning

B3C: A Minimalist Approach to Offline Multi-Agent Reinforcement Learning

2025-01-30 · Woojun Kim, Katia Sycara

Overestimation arising from selecting unseen actions during policy evaluation is a major challenge in offline reinforcement learning (RL). A minimalist approach in the single-agent setting -- adding behavior cloning (BC)…

Multi-agent Reinforcement Learning Reinforcement Learning (RL)

A Minimalist Approach to Offline Reinforcement Learning

2021-06-12 · NeurIPS 2021 12 · Scott Fujimoto, Shixiang Shane Gu

Offline reinforcement learning (RL) defines the task of learning from a fixed batch of data. Due to errors in value estimation from out-of-distribution actions, most offline RL algorithms take the approach of constrainin…

Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

120 Minutes and a Laptop: Minimalist Image-goal Navigation via Unsupervised Exploration and Offline RL

2026-03-27 · Xiaoming Liu, Borong Zhang, Qingbiao Li, Steven Morad arxiv

The prevailing paradigm for image-goal visual navigation often assumes access to large-scale datasets, substantial pretraining, and significant computational resources. In this work, we challenge this assumption. We show…

Computational Efficiency Reinforcement Learning Visual Navigation Offline RL

A Clean Slate for Offline Reinforcement Learning

2025-04-15 · Matthew Thomas Jackson, Uljad Berdica, Jarek Liesen, Shimon Whiteson 외

Progress in offline reinforcement learning (RL) has been impeded by ambiguous problem definitions and entangled algorithmic designs, resulting in inconsistent implementations, insufficient ablations, and unfair evaluatio…