A Forget-and-Grow Strategy for Deep Reinforcement Learning Scaling in Continuous Control

2025-07-03 · Zilin Kang, Chenyuan Hu, Yu Luo, Zhecheng Yuan, Ruijie Zheng, Huazhe Xu arxiv

Deep reinforcement learning for continuous control has recently achieved impressive progress. However, existing methods often suffer from primacy bias, a tendency to overfit early experiences stored in the replay buffer, which limits an RL agent's sample efficiency and generalizability. In contrast, humans are less susceptible to such bias, partly due to infantile amnesia, where the formation of new neurons disrupts early memory traces, leading to the forgetting of initial experiences. Inspired by this dual processes of forgetting and growing in neuroscience, in this paper, we propose Forget and Grow (FoG), a new deep RL algorithm with two mechanisms introduced. First, Experience Replay Decay (ER Decay) "forgetting early experience", which balances memory by gradually reducing the influence of early experiences. Second, Network Expansion, "growing neural capacity", which enhances agents' capability to exploit the patterns of existing data by dynamically adding new parameters during training. Empirical results on four major continuous control benchmarks with more than 40 tasks demonstrate the superior performance of FoG against SoTA existing deep RL algorithms, including BRO, SimBa, and TD-MPC2.

📄 PDF Abstract BibTeX arXiv:2507.02712

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning Continuous Control

Grow, Assess, Compress: Adaptive Backbone Scaling for Memory-Efficient Class Incremental Learning

2026-03-09 · Adrian Garcia-Castañeda, Jon Irureta, Jon Imaz, Aizea Lojo arxiv

Class Incremental Learning (CIL) poses a fundamental challenge: maintaining a balance between the plasticity required to learn new tasks and the stability needed to prevent catastrophic forgetting. While expansion-based …

Class Incremental Learning

Self-Composing Policies for Scalable Continual Reinforcement Learning

2025-06-04 · Mikel Malagón, Josu Ceberio, Jose A. Lozano

This work introduces a growable and modular neural network architecture that naturally avoids catastrophic forgetting and interference in continual reinforcement learning. The structure of each module allows the selectiv…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1

Continual Adversarial Reinforcement Learning (CARL) of False Data Injection detection: forgetting and explainability

2024-11-15 · Pooja Aslami, Kejun Chen, Timothy M. Hansen, Malik Hassanaly

False data injection attacks (FDIAs) on smart inverters are a growing concern linked to increased renewable energy production. While data-based FDIA detection methods are also actively developed, we show that they remain…

Continual Learning Reinforcement Learning (RL)

Taming the Implicit: Dual-Channel Risk-Aware Reinforcement Fine-Tuning for Continual Multimodal Post-Training

2026-08-04 · Yibei Liu, Jiajun Chen, Qianle Zhang, Tangyue Jin 외 arxiv

Reinforcement fine-tuning (RFT) is widely believed to inherently resist catastrophic forgetting in continual post-training of multimodal large language models. Under pronounced task distributional shifts, however, forget…

Progressive Continual Learning for Spoken Keyword Spotting

2022-01-29 · Yizheng Huang, Nana Hou, Nancy F. Chen

Catastrophic forgetting is a thorny challenge when updating keyword spotting (KWS) models after deployment. To tackle such challenges, we propose a progressive continual learning strategy for small-footprint spoken keywo…

Continual Learning Keyword Spotting