Maintaining Plasticity in Continual Learning via Regenerative Regularization
In continual learning, plasticity refers to the ability of an agent to quickly adapt to new information. Neural networks are known to lose plasticity when processing non-stationary data streams. In this paper, we propose L2 Init, a simple approach for maintaining plasticity by incorporating in the loss function L2 regularization toward initial parameters. This is very similar to standard L2 regularization (L2), the only difference being that L2 regularizes toward the origin. L2 Init is simple to implement and requires selecting only a single hyper-parameter. The motivation for this method is the same as that of methods that reset neurons or parameter values. Intuitively, when recent losses are insensitive to particular parameters, these parameters should drift toward their initial values. This prepares parameters to adapt quickly to new tasks. On problems representative of different types of nonstationarity in continual supervised learning, we demonstrate that L2 Init most consistently mitigates plasticity loss compared to previously proposed approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningL2 RegularizationSimilar Papers 제목 키워드 기반
A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning
Continual learning with deep neural networks presents challenges distinct from both the fixed-dataset and convex continual learning regimes. One such challenge is plasticity loss, wherein a neural network trained in an o…
Continual LearningDeep Reinforcement LearningMontezuma's RevengeReinforcement Learning (RL)Maintaining Plasticity in Deep Continual Learning
Modern deep-learning systems are specialized to problem settings in which training occurs once and then never again, as opposed to continual-learning settings in which training occurs continually. If deep-learning system…
Binary ClassificationContinual LearningL2 RegularizationContinually Learning Self-Supervised Representations with Projected Functional Regularization
Recent self-supervised learning methods are able to learn high-quality image representations and are closing the gap with supervised approaches. However, these methods are unable to acquire new knowledge incrementally --…
Continual LearningIncremental LearningSelf-Supervised LearningSpectral Collapse Drives Loss of Plasticity in Deep Continual Learning
We investigate why deep neural networks suffer from loss of plasticity in continual learning, and thus fail to learn new tasks without reinitializing parameters. We show that this failure is preceded by Hessian spectral …
Reinforcement LearningContinual LearningPreserving Plasticity in Continual Learning via Dynamical Isometry
Continual training of deep neural networks under non-stationarity often leads to a progressive loss of plasticity, eventually limiting further learning. We relate plasticity to the empirical Neural Tangent Kernel, and id…
Continual Learning