Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification
Few-shot text classification aims to classify the text under the few-shot scenario. Most of the previous methods adopt optimization-based meta learning to obtain task distribution. However, due to the neglect of matching between the few amount of samples and complicated models, as well as the distinction between useful and useless task features, these methods suffer from the overfitting issue. To address this issue, we propose a novel Adaptive Meta-learner via Gradient Similarity (AMGS) method to improve the model generalization ability to a new task. Specifically, the proposed AMGS alleviates the overfitting based on two aspects: (i) acquiring the potential semantic representation of samples and improving model generalization through the self-supervised auxiliary task in the inner loop, (ii) leveraging the adaptive meta-learner via gradient similarity to add constraints on the gradient obtained by base-learner in the outer loop. Moreover, we make a systematic analysis of the influence of regularization on the entire framework. Experimental results on several benchmarks demonstrate that the proposed AMGS consistently improves few-shot text classification performance compared with the state-of-the-art optimization-based meta-learning approaches.
Code (1)
Tasks
Few-Shot Text ClassificationMeta-Learningtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Layer-Wise Adaptive Updating for Few-Shot Image Classification
Few-shot image classification (FSIC), which requires a model to recognize new categories via learning from few images of these categories, has attracted lots of attention. Recently, meta-learning based methods have been …
ClassificationFew-Shot Image ClassificationGeneral Classificationimage-classification+2Cooperative Meta-Learning with Gradient Augmentation
Model agnostic meta-learning (MAML) is one of the most widely used gradient-based meta-learning, consisting of two optimization loops: an inner loop and outer loop. MAML learns the new task from meta-initialization param…
Few-Shot Image Classificationimage-classificationImage ClassificationMeta-Learning+1Meta-AdaM: An Meta-Learned Adaptive Optimizer with Momentum for Few-Shot Learning
We introduce Meta-AdaM, a meta-learned adaptive optimizer with momentum, designed for few-shot learning tasks that pose significant challenges to deep learning models due to the limited number of labeled examples. Meta-l…
MetaSLRCL: A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification
Due to the lack of labeled data in many realistic scenarios, a number of few-shot learning methods for text classification have been proposed, among which the meta learning based ones have recently attracted much attenti…
Few-Shot LearningFew-Shot Text ClassificationMeta-Learningtext-classification+1A Self-Adaptive Learning Rate and Curriculum Learning Based Framework for Few-Shot Text Classification
Due to the lack of labeled data in many realistic scenarios, a number of few-shot learning methods for text classification have been proposed, among which the meta learning based ones have recently attracted much attenti…
Few-Shot LearningFew-Shot Text ClassificationMeta-Learningtext-classification+1