Attentive Task-Agnostic Meta-Learning for Few-Shot Text Classification
Current deep learning based text classification methods are limited by their ability to achieve fast learning and generalization when the data is scarce. We address this problem by integrating a meta-learning procedure that uses the knowledge learned across many tasks as an inductive bias towards better natural language understanding. Inspired by the Model-Agnostic Meta-Learning framework (MAML), we introduce the Attentive Task-Agnostic Meta-Learning (ATAML) algorithm for text classification. The proposed ATAML is designed to encourage task-agnostic representation learning by way of task-agnostic parameterization and facilitate task-specific adaptation via attention mechanisms. We provide evidence to show that the attention mechanism in ATAML has a synergistic effect on learning performance. Our experimental results reveal that, for few-shot text classification tasks, gradient-based meta-learning approaches ourperform popular transfer learning methods. In comparisons with models trained from random initialization, pretrained models and meta trained MAML, our proposed ATAML method generalizes better on single-label and multi-label classification tasks in miniRCV1 and miniReuters-21578 datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationFew-Shot Text ClassificationGeneral ClassificationInductive BiasMeta-LearningMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONNatural Language UnderstandingRepresentation Learningtext-classificationText ClassificationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
On the Importance of Attention in Meta-Learning for Few-Shot Text Classification
Current deep learning based text classification methods are limited by their ability to achieve fast learning and generalization when the data is scarce. We address this problem by integrating a meta-learning procedure t…
ClassificationFew-Shot Text ClassificationGeneral ClassificationInductive Bias+7Attentive Weights Generation for Few Shot Learning via Information Maximization
Few shot image classification aims at learning a classifier from limited labeled data. Generating the classification weights has been applied in many meta-learning methods for few shot image classification due to its sim…
ClassificationFew-Shot Image ClassificationFew-Shot LearningGeneral Classification+3LEA: Meta Knowledge-Driven Self-Attentive Document Embedding for Few-Shot Text Classification
Text classification has achieved great success with the prosperity of deep learning and pre-trained language models. However, we often encounter labeled data deficiency problems in real-world text-classification tasks. T…
ClassificationDocument EmbeddingFew-Shot LearningFew-Shot Text Classification+3Revisiting Local Descriptor for Improved Few-Shot Classification
Few-shot classification studies the problem of quickly adapting a deep learner to understanding novel classes based on few support images. In this context, recent research efforts have been aimed at designing more and mo…
ClassificationDecision MakingGeneral ClassificationMeta-Learning with Context-Agnostic Initialisations
Meta-learning approaches have addressed few-shot problems by finding initialisations suited for fine-tuning to target tasks. Often there are additional properties within training data (which we refer to as context), not …
Meta-Learning