MAML is a Noisy Contrastive Learner in Classification
Model-agnostic meta-learning (MAML) is one of the most popular and widely adopted meta-learning algorithms, achieving remarkable success in various learning problems. Yet, with the unique design of nested inner-loop and outer-loop updates, which govern the task-specific and meta-model-centric learning, respectively, the underlying learning objective of MAML remains implicit and thus impedes a more straightforward understanding of it. In this paper, we provide a new perspective of the working mechanism of MAML. We discover that MAML is analogous to a meta-learner using a supervised contrastive objective. The query features are pulled towards the support features of the same class and against those of different classes. Such contrastiveness is experimentally verified via an analysis based on the cosine similarity. Moreover, we reveal that vanilla MAML has an undesirable interference term originating from the random initialization and the cross-task interaction. We thus propose a simple but effective technique, zeroing trick, to alleviate the interference. Extensive experiments are conducted on both mini-ImageNet and Omniglot datasets to demonstrate the consistent improvement brought by our proposed method, validating its effectiveness.
Code (1)
Tasks
ClassificationFew-Shot LearningMeta-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
BatMan-CLR: Making Few-shots Meta-Learners Resilient Against Label Noise
The negative impact of label noise is well studied in classical supervised learning yet remains an open research question in meta-learning. Meta-learners aim to adapt to unseen learning tasks by learning a good initial m…
Meta-LearningStateless Neural Meta-Learning using Second-Order Gradients
Deep learning typically requires large data sets and much compute power for each new problem that is learned. Meta-learning can be used to learn a good prior that facilitates quick learning, thereby relaxing these requir…
image-classificationImage ClassificationMeta-LearningA Nested Bi-level Optimization Framework for Robust Few Shot Learning
Model-Agnostic Meta-Learning (MAML), a popular gradient-based meta-learning framework, assumes that the contribution of each task or instance to the meta-learner is equal. Hence, it fails to address the domain shift betw…
Few-Shot LearningMeta-LearningQ-MAML: Quantum Model-Agnostic Meta-Learning for Variational Quantum Algorithms
In the Noisy Intermediate-Scale Quantum (NISQ) era, using variational quantum algorithms (VQAs) to solve optimization problems has become a key application. However, these algorithms face significant challenges, such as …
Meta-LearningAuto-Meta: Automated Gradient Based Meta Learner Search
Fully automating machine learning pipelines is one of the key challenges of current artificial intelligence research, since practical machine learning often requires costly and time-consuming human-powered processes such…
BIG-bench Machine LearningMeta-LearningNeural Architecture Search