An Efficient Replay for Class-Incremental Learning with Pre-trained Models

2024-08-15 · Weimin Yin, Bin Chen adn Chunzhao Xie, Zhenhao Tan

In general class-incremental learning, researchers typically use sample sets as a tool to avoid catastrophic forgetting during continuous learning. At the same time, researchers have also noted the differences between class-incremental learning and Oracle training and have attempted to make corrections. In recent years, researchers have begun to develop class-incremental learning algorithms utilizing pre-trained models, achieving significant results. This paper observes that in class-incremental learning, the steady state among the weight guided by each class center is disrupted, which is significantly correlated with catastrophic forgetting. Based on this, we propose a new method to overcoming forgetting . In some cases, by retaining only a single sample unit of each class in memory for replay and applying simple gradient constraints, very good results can be achieved. Experimental results indicate that under the condition of pre-trained models, our method can achieve competitive performance with very low computational cost and by simply using the cross-entropy loss.

📄 PDF Abstract BibTeX arXiv:2408.08084

Code (0)

등록된 구현이 없습니다.

Tasks

class-incremental learning Class Incremental Learning Incremental Learning

Dual-Teacher Class-Incremental Learning With Data-Free Generative Replay

2021-06-17 · Yoojin Choi, Mostafa El-Khamy, Jungwon Lee

This paper proposes two novel knowledge transfer techniques for class-incremental learning (CIL). First, we propose data-free generative replay (DF-GR) to mitigate catastrophic forgetting in CIL by using synthetic sample…

class-incremental learning Class Incremental Learning Incremental Learning Knowledge Distillation+1

Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection

2022-09-20 · Tobias Kalb, Björn Mauthe, Jürgen Beyerer

Continual learning for Semantic Segmentation (CSS) is a rapidly emerging field, in which the capabilities of the segmentation model are incrementally improved by learning new classes or new domains. A central challenge i…

Continual Learning Continual Semantic Segmentation Incremental Learning Segmentation+1

Generative Feature-driven Image Replay for Continual Learning

2021-06-09 · Kevin Thandiackal, Tiziano Portenier, Andrea Giovannini, Maria Gabrani 외

Neural networks are prone to catastrophic forgetting when trained incrementally on different tasks. Popular incremental learning methods mitigate such forgetting by retaining a subset of previously seen samples and repla…

Class Incremental Learning Continual Learning Incremental Learning

Class-Wise Buffer Management for Incremental Object Detection: An Effective Buffer Training Strategy

2023-12-14 · Junsu Kim, Sumin Hong, Chanwoo Kim, Jihyeon Kim 외

Class incremental learning aims to solve a problem that arises when continuously adding unseen class instances to an existing model This approach has been extensively studied in the context of image classification; howev…

class-incremental learning Class Incremental Learning image-classification Image Classification+6

Class-Incremental Learning using Diffusion Model for Distillation and Replay

2023-06-30 · Quentin Jodelet, Xin Liu, Yin Jun Phua, Tsuyoshi Murata

Class-incremental learning aims to learn new classes in an incremental fashion without forgetting the previously learned ones. Several research works have shown how additional data can be used by incremental models to he…