Continual Learning with Neuron Activation Importance
Continual learning is a concept of online learning with multiple sequential tasks. One of the critical barriers of continual learning is that a network should learn a new task keeping the knowledge of old tasks without access to any data of the old tasks. In this paper, we propose a neuron activation importance-based regularization method for stable continual learning regardless of the order of tasks. We conduct comprehensive experiments on existing benchmark data sets to evaluate not just the stability and plasticity of our method with improved classification accuracy also the robustness of the performance along the changes of task order.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningSimilar Papers 제목 키워드 기반
Continual learning with neural activation importance
Continual learning is a concept of online learning along with multiple sequential tasks. One of the critical barriers of continual learning is that a network should learn a new task keeping the knowledge of old tasks wit…
Continual LearningShapley Neuron Values for Continual Learning: Which Neurons Matter Most?
Continual learning enables neural networks to learn tasks sequentially without forgetting previously acquired knowledge. However, neural networks suffer from catastrophic forgetting, where learning new tasks degrades per…
Class Incremental LearningContinual LearningGPrune-LLM: Generalization-Aware Structured Pruning for Large Language Models
Structured pruning is widely used to compress large language models (LLMs), yet its effectiveness depends heavily on neuron importance estimation. Most existing methods estimate neuron importance from activation statisti…
Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning
Deep reinforcement learning (RL) agents frequently suffer from neuronal activity loss, which impairs their ability to adapt to new data and learn continually. A common method to quantify and address this issue is the tau…
Deep Reinforcement LearningMuJoCoReinforcement Learning (RL)Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning
Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered. Existing CL approaches often keep a buffer of previous…
Continual LearningKnowledge DistillationOpen Knowledge Graph Canonicalization