Deep Reinforcement Learning with Plasticity Injection

2023-05-24 · NeurIPS 2023 11

A growing body of evidence suggests that neural networks employed in deep reinforcement learning (RL) gradually lose their plasticity, the ability to learn from new data; however, the analysis and mitigation of this phenomenon is hampered by the complex relationship between plasticity, exploration, and performance in RL. This paper introduces plasticity injection, a minimalistic intervention that increases the network plasticity without changing the number of trainable parameters or biasing the predictions. The applications of this intervention are two-fold: first, as a diagnostic tool $\unicode{x2014}$ if injection increases the performance, we may conclude that an agent's network was losing its plasticity. This tool allows us to identify a subset of Atari environments where the lack of plasticity causes performance plateaus, motivating future studies on understanding and combating plasticity loss. Second, plasticity injection can be used to improve the computational efficiency of RL training if the agent has to re-learn from scratch due to exhausted plasticity or by growing the agent's network dynamically without compromising performance. The results on Atari show that plasticity injection attains stronger performance compared to alternative methods while being computationally efficient.

📄 PDF Abstract BibTeX arXiv:2305.15555

Code (0)

등록된 구현이 없습니다.

Tasks

Computational EfficiencyDeep Reinforcement LearningDiagnosticreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Stagnant Neuron: Towards Understanding the Plasticity Loss in Multi-Agent Reinforcement Learning Value Factorization Methods

2026-06-24 · Zhengzhu Liu, Zeming Gao, Haoyuan Qin, Jiawei Hu 외 arxiv

Multi-Agent Reinforcement Learning (MARL) value factorization methods can suffer from a loss of plasticity, gradually failing to adapt when transferring to new task instances. We trace this issue to stagnant neurons, uni…

Multi-agent Reinforcement Learning

The Rank and Gradient Lost in Non-stationarity: Sample Weight Decay for Mitigating Plasticity Loss in Reinforcement Learning

2026-04-02 · Zihao Wu, Hongyao Tang, Yi Ma, Jiashun Liu 외 arxiv

Deep reinforcement learning (RL) suffers from plasticity loss severely due to the nature of non-stationarity, which impairs the ability to adapt to new data and learn continually. Unfortunately, our understanding of how …

Reinforcement Learning

Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learning

2026-05-14 · Oubo Ma, Ruixiao Lin, Yang Dai, Jiahao Chen 외 arxiv

Extensive research has highlighted the severe threats posed by backdoor attacks to deep reinforcement learning (DRL). However, prior studies primarily focus on vanilla scenarios, while plasticity interventions have emerg…

Reinforcement Learning

Preserving Plasticity in Continual Learning with Adaptive Linearity Injection

2025-05-14 · Seyed Roozbeh Razavi Rohani, Khashayar Khajavi, Wesley Chung, Mo Chen 외

Loss of plasticity in deep neural networks is the gradual reduction in a model's capacity to incrementally learn and has been identified as a key obstacle to learning in non-stationary problem settings. Recent work has s…

class-incremental learning Class Incremental Learning Continual Learning Incremental Learning+1

Sustaining Plasticity via Learnable Wavelet Activations in Continual Learning

2026-08-13 · Zeyang Zhang, Tieliang Gong, Junyan Lu, Weizhan Zhang arxiv

Plasticity loss has emerged as a critical challenge in continual learning that significantly hinders the acquisition of sequential tasks. While optimizing activation designs offers a potential solution, current fixed-for…

Continual Learning