Memory-Based Label-Text Tuning for Few-Shot Class-Incremental Learning
Few-shot class-incremental learning(FSCIL) focuses on designing learning algorithms that can continually learn a sequence of new tasks from a few samples without forgetting old ones. The difficulties are that training on a sequence of limited data from new tasks leads to severe overfitting issues and causes the well-known catastrophic forgetting problem. Existing researches mainly utilize the image information, such as storing the image knowledge of previous tasks or limiting classifiers updating. However, they ignore analyzing the informative and less noisy text information of class labels. In this work, we propose leveraging the label-text information by adopting the memory prompt. The memory prompt can learn new data sequentially, and meanwhile store the previous knowledge. Furthermore, to optimize the memory prompt without undermining the stored knowledge, we propose a stimulation-based training strategy. It optimizes the memory prompt depending on the image embedding stimulation, which is the distribution of the image embedding elements. Experiments show that our proposed method outperforms all prior state-of-the-art approaches, significantly mitigating the catastrophic forgetting and overfitting problems.
Code (0)
등록된 구현이 없습니다.
Tasks
class-incremental learningClass Incremental LearningFew-Shot Class-Incremental LearningIncremental LearningSimilar Papers 제목 키워드 기반
The Benefits of Label-Description Training for Zero-Shot Text Classification
Pretrained language models have improved zero-shot text classification by allowing the transfer of semantic knowledge from the training data in order to classify among specific label sets in downstream tasks. We propose …
Classificationdomain classificationtext-classificationText Classification+3Few-Shot Learning with Siamese Networks and Label Tuning
We study the problem of building text classifiers with little or no training data, commonly known as zero and few-shot text classification. In recent years, an approach based on neural textual entailment models has been …
Few-Shot LearningFew-Shot Text ClassificationNatural Language Inferencetext-classification+1Few-Shot Learning with Siamese Networks and Label Tuning
We study the problem of building text classifiers with little or no training data, commonly known as zero and few-shot text classification. In recent years, an approach based on neural textual entailment models has been …
Few-Shot LearningFew-Shot Text ClassificationNatural Language Inferencetext-classification+1LM-BFF-MS: Improving Few-Shot Fine-tuning of Language Models based on Multiple Soft Demonstration Memory
LM-BFF (CITATION) achieves significant few-shot performance by using auto-generated prompts and adding demonstrations similar to an input example. To improve the approach of LM-BFF, this paper proposes LM-BFF-MS—better f…
MRPCSST-2Word EmbeddingsZero-Shot Text Classification via Self-Supervised Tuning
Existing solutions to zero-shot text classification either conduct prompting with pre-trained language models, which is sensitive to the choices of templates, or rely on large-scale annotated data of relevant tasks for m…
ClassificationSelf-Supervised LearningSentenceSentiment Analysis+4