Detachedly Learn a Classifier for Class-Incremental Learning

2023-02-23 · Ziheng Li, Shibo Jie, Zhi-Hong Deng

In continual learning, model needs to continually learn a feature extractor and classifier on a sequence of tasks. This paper focuses on how to learn a classifier based on a pretrained feature extractor under continual learning setting. We present an probabilistic analysis that the failure of vanilla experience replay (ER) comes from unnecessary re-learning of previous tasks and incompetence to distinguish current task from the previous ones, which is the cause of knowledge degradation and prediction bias. To overcome these weaknesses, we propose a novel replay strategy task-aware experience replay. It rebalances the replay loss and detaches classifier weight for the old tasks from the update process, by which the previous knowledge is kept intact and the overfitting on episodic memory is alleviated. Experimental results show our method outperforms current state-of-the-art methods.

📄 PDF Abstract BibTeX arXiv:2302.11730

Code (0)

등록된 구현이 없습니다.

Tasks

class-incremental learning Class Incremental Learning Continual Learning Incremental Learning

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

EILearn: Learning Incrementally Using Previous Knowledge Obtained From an Ensemble of Classifiers

2019-02-08 · Shivang Agarwal, C. Ravindranath Chowdary, Shripriya Maheshwari

We propose an algorithm for incremental learning of classifiers. The proposed method enables an ensemble of classifiers to learn incrementally by accommodating new training data. We use an effective mechanism to overcome…

Incremental Learning

More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning

2020-08-01 · ECCV 2020 8 · Yu Liu, Sarah Parisot, Gregory Slabaugh, Xu Jia 외

Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning","Overcoming catastrophic forgetting in neural networks is a long-standing and core research objective for incremental learning. Notable studi…

Incremental Learning

Teamwork Is Not Always Good: An Empirical Study of Classifier Drift in Class-incremental Information Extraction

2023-05-26 · Minqian Liu, Lifu Huang

Class-incremental learning (CIL) aims to develop a learning system that can continually learn new classes from a data stream without forgetting previously learned classes. When learning classes incrementally, the classif…

class-incremental learning Class Incremental Learning Incremental Learning

Distillation Techniques for Pseudo-rehearsal Based Incremental Learning

2018-07-08 · Haseeb Shah, Khurram Javed, Faisal Shafait

The ability to learn from incrementally arriving data is essential for any life-long learning system. However, standard deep neural networks forget the knowledge about the old tasks, a phenomenon called catastrophic forg…

Incremental Learning Knowledge Distillation

Few-Shot Incremental Learning with Continually Evolved Classifiers

2021-04-07 · CVPR 2021 1 · Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng 외

Few-shot class-incremental learning (FSCIL) aims to design machine learning algorithms that can continually learn new concepts from a few data points, without forgetting knowledge of old classes. The difficulty lies in t…

class-incremental learning Class Incremental Learning Few-Shot Class-Incremental Learning Incremental Learning