More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning
Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning","Overcoming catastrophic forgetting in neural networks is a long-standing and core research objective for incremental learning. Notable studies have shown regularization strategies enable the network to remember previously acquired knowledge devoid of heavy forgetting. Since those regularization strategies are mostly associated with classifier outputs, we propose a MUlti-Classifier (MUC) incremental learning paradigm that integrates an ensemble of auxiliary classifiers to estimate more effective regularization constraints. Additionally, we extend two common methods, focusing on parameter and activation regularization, from the conventional single-classifier paradigm to MUC. Our classifier ensemble promotes regularizing network parameters or activations when moving to learn the next task. Under the setting of task-agnostic evaluation, our experimental results on CIFAR-100 and Tiny ImageNet incremental benchmarks show that our method outperforms other baselines. Specifically, MUC obtains 3%-5% accuracy boost and 4%-5% decline of forgetting ratio, compared with MAS and LwF. Our code is available at https://github.com/Liuy8/MUC.
Code (1)
Tasks
Incremental LearningSimilar Papers 제목 키워드 기반
Balanced Supervised Contrastive Learning for Few-Shot Class-Incremental Learning
Few-shot class-incremental learning (FSCIL) presents the primary challenge of balancing underfitting to a new session's task and forgetting the tasks from previous sessions. To address this challenge, we develop a simple…
class-incremental learningClass Incremental LearningContrastive LearningFew-Shot Class-Incremental Learning+1Replaying pre-training data improves fine-tuning
To obtain a language model for a target domain (e.g. math), the current paradigm is to pre-train on a vast amount of generic web text and then fine-tune on the relatively limited amount of target data. Typically, generic…
Class-Incremental Few-Shot Object Detection
Conventional detection networks usually need abundant labeled training samples, while humans can learn new concepts incrementally with just a few examples. This paper focuses on a more challenging but realistic class-inc…
ClusteringFew-Shot Object DetectionKnowledge DistillationObject+2Fixed Random Classifier Rearrangement for Continual Learning
With the explosive growth of data, continual learning capability is increasingly important for neural networks. Due to catastrophic forgetting, neural networks inevitably forget the knowledge of old tasks after learning …
Binary ClassificationContinual LearningBayesian Parameter-Efficient Fine-Tuning for Overcoming Catastrophic Forgetting
We are motivated primarily by the adaptation of text-to-speech synthesis models; however we argue that more generic parameter-efficient fine-tuning (PEFT) is an appropriate framework to do such adaptation. Nevertheless, …
Language ModelingLanguage Modellingparameter-efficient fine-tuningSpeech Synthesis+3