The Dormant Neuron Phenomenon in Deep Reinforcement Learning
In this work we identify the dormant neuron phenomenon in deep reinforcement learning, where an agent's network suffers from an increasing number of inactive neurons, thereby affecting network expressivity. We demonstrate the presence of this phenomenon across a variety of algorithms and environments, and highlight its effect on learning. To address this issue, we propose a simple and effective method (ReDo) that Recycles Dormant neurons throughout training. Our experiments demonstrate that ReDo maintains the expressive power of networks by reducing the number of dormant neurons and results in improved performance.
Code (3)
Tasks
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Mastering Continual Reinforcement Learning through Fine-Grained Sparse Network Allocation and Dormant Neuron Exploration
Continual Reinforcement Learning (CRL) is essential for developing agents that can learn, adapt, and accumulate knowledge over time. However, a fundamental challenge persists as agents must strike a delicate balance betw…
DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization
Visual reinforcement learning (RL) has shown promise in continuous control tasks. Despite its progress, current algorithms are still unsatisfactory in virtually every aspect of the performance such as sample efficiency, …
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+1Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning
Deep reinforcement learning (RL) agents frequently suffer from neuronal activity loss, which impairs their ability to adapt to new data and learn continually. A common method to quantify and address this issue is the tau…
Deep Reinforcement LearningMuJoCoReinforcement Learning (RL)Neuroplastic Expansion in Deep Reinforcement Learning
The loss of plasticity in learning agents, analogous to the solidification of neural pathways in biological brains, significantly impedes learning and adaptation in reinforcement learning due to its non-stationary nature…
Deep Reinforcement LearningMuJoCoreinforcement-learningReinforcement LearningCondensation of nucleoid in Escherichia coli cell as a result of prolonged starvation
Electron microscopy and X-ray diffraction studies of dormant E. coli cells revealed several forms of nucleoid condensation: quasi - nanocrystalline, quasi -liquid crystalline (or spore-like) and folded nucleosome - like …