Covariance-based Space Regularization for Few-shot Class Incremental Learning

2024-11-02 · Yijie Hu, Guanyu Yang, Zhaorui Tan, Xiaowei Huang, Kaizhu Huang, Qiu-Feng Wang

Few-shot Class Incremental Learning (FSCIL) presents a challenging yet realistic scenario, which requires the model to continually learn new classes with limited labeled data (i.e., incremental sessions) while retaining knowledge of previously learned base classes (i.e., base sessions). Due to the limited data in incremental sessions, models are prone to overfitting new classes and suffering catastrophic forgetting of base classes. To tackle these issues, recent advancements resort to prototype-based approaches to constrain the base class distribution and learn discriminative representations of new classes. Despite the progress, the limited data issue still induces ill-divided feature space, leading the model to confuse the new class with old classes or fail to facilitate good separation among new classes. In this paper, we aim to mitigate these issues by directly constraining the span of each class distribution from a covariance perspective. In detail, we propose a simple yet effective covariance constraint loss to force the model to learn each class distribution with the same covariance matrix. In addition, we propose a perturbation approach to perturb the few-shot training samples in the feature space, which encourages the samples to be away from the weighted distribution of other classes. Regarding perturbed samples as new class data, the classifier is forced to establish explicit boundaries between each new class and the existing ones. Our approach is easy to integrate into existing FSCIL approaches to boost performance. Experiments on three benchmarks validate the effectiveness of our approach, achieving a new state-of-the-art performance of FSCIL.

📄 PDF Abstract BibTeX arXiv:2411.01172

Code (1)

tambourine666/Covariance-Space-Regularize 공식 구현 pytorch

Tasks

class-incremental learning Class Incremental Learning Few-Shot Class-Incremental Learning Incremental Learning

Methods 이 논문이 사용한 방법론

BASE 설명 없음

Subspace Regularizers for Few-Shot Class Incremental Learning

2021-10-13 · ICLR 2022 4 · Afra Feyza Akyürek, Ekin Akyürek, Derry Tanti Wijaya, Jacob Andreas

Few-shot class incremental learning -- the problem of updating a trained classifier to discriminate among an expanded set of classes with limited labeled data -- is a key challenge for machine learning systems deployed i…

class-incremental learning Class Incremental Learning Continual Learning Few-Shot Class-Incremental Learning+4

Incremental few-shot learning via vector quantization in deep embedded space

2021-01-01 · ICLR 2021 1 · Kuilin Chen, Chi-Guhn Lee

The capability of incrementally learning new tasks without forgetting old ones is a challenging problem due to catastrophic forgetting. This challenge becomes greater when novel tasks contain very few labelled training s…

class-incremental learning Class Incremental Learning Few-Shot Class-Incremental Learning Few-Shot Learning+3

Incremental Few-Shot Object Detection for Robotics

2020-05-06 · Yiting Li, Haiyue Zhu, Sichao Tian, Fan Feng 외

Incremental few-shot learning is highly expected for practical robotics applications. On one hand, robot is desired to learn new tasks quickly and flexibly using only few annotated training samples; on the other hand, su…

Classification Continual Learning Few-Shot Learning Few-Shot Object Detection+5

Few-shot Class-Incremental Learning via Generative Co-Memory Regularization

2026-01-12 · Kexin Bao, Yong Li, Dan Zeng, Shiming Ge arxiv

Few-shot class-incremental learning (FSCIL) aims to incrementally learn models from a small amount of novel data, which requires strong representation and adaptation ability of models learned under few-example supervisio…

Few-Shot Class-Incremental Learning Domain Adaptation

Learnable Distribution Calibration for Few-Shot Class-Incremental Learning

2022-10-01 · Binghao Liu, Boyu Yang, Lingxi Xie, Ren Wang 외

Few-shot class-incremental learning (FSCIL) faces challenges of memorizing old class distributions and estimating new class distributions given few training samples. In this study, we propose a learnable distribution cal…

class-incremental learning Class Incremental Learning Few-Shot Class-Incremental Learning Few-Shot Learning+2