Balanced Supervised Contrastive Learning for Few-Shot Class-Incremental Learning
Few-shot class-incremental learning (FSCIL) presents the primary challenge of balancing underfitting to a new session's task and forgetting the tasks from previous sessions. To address this challenge, we develop a simple yet powerful learning scheme that integrates effective methods for each core component of the FSCIL network, including the feature extractor, base session classifiers, and incremental session classifiers. In feature extractor training, our goal is to obtain balanced generic representations that benefit both current viewable and unseen or past classes. To achieve this, we propose a balanced supervised contrastive loss that effectively balances these two objectives. In terms of classifiers, we analyze and emphasize the importance of unifying initialization methods for both the base and incremental session classifiers. Our method demonstrates outstanding ability for new task learning and preventing forgetting on CUB200, CIFAR100, and miniImagenet datasets, with significant improvements over previous state-of-the-art methods across diverse metrics. We conduct experiments to analyze the significance and rationale behind our approach and visualize the effectiveness of our representations on new tasks. Furthermore, we conduct diverse ablation studies to analyze the effects of each module.
Code (0)
등록된 구현이 없습니다.
Tasks
class-incremental learningClass Incremental LearningContrastive LearningFew-Shot Class-Incremental LearningIncremental LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Class Incremental Fault Diagnosis under Limited Fault Data via Supervised Contrastive Knowledge Distillation
Class-incremental fault diagnosis requires a model to adapt to new fault classes while retaining previous knowledge. However, limited research exists for imbalanced and long-tailed data. Extracting discriminative feature…
Fault DiagnosisKnowledge DistillationRepresentation LearningExemplar-Based Contrastive Self-Supervised Learning with Few-Shot Class Incremental Learning
Humans are capable of learning new concepts from only a few (labeled) exemplars, incrementally and continually. This happens within the context that we can differentiate among the exemplars, and between the exemplars and…
class-incremental learningClass Incremental LearningData AugmentationFew-Shot Class-Incremental Learning+3Towards Robust Few-shot Class Incremental Learning in Audio Classification using Contrastive Representation
In machine learning applications, gradual data ingress is common, especially in audio processing where incremental learning is vital for real-time analytics. Few-shot class-incremental learning addresses challenges arisi…
Audio Classificationclass-incremental learningClass Incremental LearningContrastive Learning+2SCLIFD:Supervised Contrastive Knowledge Distillation for Incremental Fault Diagnosis under Limited Fault Data
Intelligent fault diagnosis has made extraordinary advancements currently. Nonetheless, few works tackle class-incremental learning for fault diagnosis under limited fault data, i.e., imbalanced and long-tailed fault dia…
class-incremental learningClass Incremental LearningFault DiagnosisIncremental Learning+2OrCo: Towards Better Generalization via Orthogonality and Contrast for Few-Shot Class-Incremental Learning
Few-Shot Class-Incremental Learning (FSCIL) introduces a paradigm in which the problem space expands with limited data. FSCIL methods inherently face the challenge of catastrophic forgetting as data arrives incrementally…
class-incremental learningClass Incremental LearningContrastive LearningFew-Shot Class-Incremental Learning+1