Supervised Momentum Contrastive Learning for Few-Shot Classification
Few-shot learning aims to transfer information from one task to enable generalization on novel tasks given a few examples. This information is present both in the domain and the class labels. In this work we investigate the complementary roles of these two sources of information by combining instance-discriminative contrastive learning and supervised learning in a single framework called Supervised Momentum Contrastive learning (SUPMOCO). Our approach avoids a problem observed in supervised learning where information in images not relevant to the task is discarded, which hampers their generalization to novel tasks. We show that (self-supervised) contrastive learning and supervised learning are mutually beneficial, leading to a new state-of-the-art on the META-DATASET - a recently introduced benchmark for few-shot learning. Our method is based on a simple modification of MOCO and scales better than prior work on combining supervised and self-supervised learning. This allows us to easily combine data from multiple domains leading to further improvements.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationContrastive LearningFew-Shot LearningGeneral ClassificationModel SelectionSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CrossMoCo: Multi-modal Momentum Contrastive Learning for Point Cloud
The point cloud is a 3D geometric data that lacks a specific structure and is permutation-invariant. The applications of point clouds have gained significant attention recently in the field of vision tasks. However, most…
3D Object Classification3D Point Cloud Classification3D Point Cloud Linear ClassificationContrastive Learning+3Momentum Contrastive Pre-training for Question Answering
Existing pre-training methods for extractive Question Answering (QA) generate cloze-like queries different from natural questions in syntax structure, which could overfit pre-trained models to simple keyword matching. In…
BenchmarkingContrastive LearningExtractive Question-AnsweringNatural Questions+1MoPro: Webly Supervised Learning with Momentum Prototypes
We propose a webly-supervised representation learning method that does not suffer from the annotation unscalability of supervised learning, nor the computation unscalability of self-supervised learning. Most existing wor…
Contrastive Learningimage-classificationImage ClassificationRepresentation Learning+1mcBERT: Momentum Contrastive Learning with BERT for Zero-Shot Slot Filling
Zero-shot slot filling has received considerable attention to cope with the problem of limited available data for the target domain. One of the important factors in zero-shot learning is to make the model learn generaliz…
Contrastive Learningslot-fillingSlot FillingZero-Shot Learning+1Unsupervised Meta-Learning via Few-shot Pseudo-supervised Contrastive Learning
Unsupervised meta-learning aims to learn generalizable knowledge across a distribution of tasks constructed from unlabeled data. Here, the main challenge is how to construct diverse tasks for meta-learning without label …
Contrastive LearningCross-Domain Few-ShotMeta-LearningSelf-Supervised Learning