Momentum Contrast for Unsupervised Visual Representation Learning
We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks.
Code (44)
Tasks
Contrastive LearningImage ClassificationRepresentation LearningSelf-Supervised Image ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning
Momentum Contrast (MoCo) achieves great success for unsupervised visual representation. However, there are a lot of supervised and semi-supervised datasets, which are already labeled. To fully utilize the label annotatio…
Representation LearningUnsupervised Visual Representation Learning by Synchronous Momentum Grouping
In this paper, we propose a genuine group-level contrastive visual representation learning method whose linear evaluation performance on ImageNet surpasses the vanilla supervised learning. Two mainstream unsupervised lea…
ClusteringContrastive LearningLinear evaluationRepresentation Learning+1Masked Contrastive Representation Learning
Masked image modelling (e.g., Masked AutoEncoder) and contrastive learning (e.g., Momentum Contrast) have shown impressive performance on unsupervised visual representation learning. This work presents Masked Contrastive…
Contrastive LearningData AugmentationDecoderRepresentation LearningMomentum Contrast Speaker Representation Learning
Unsupervised representation learning has shown remarkable achievement by reducing the performance gap with supervised feature learning, especially in the image domain. In this study, to extend the technique of unsupervis…
Contrastive LearningMetric LearningRepresentation LearningSpeaker Recognition+1CO2: Consistent Contrast for Unsupervised Visual Representation Learning
Contrastive learning has been adopted as a core method for unsupervised visual representation learning. Without human annotation, the common practice is to perform an instance discrimination task: Given a query image cro…
Contrastive Learningimage-classificationImage Classificationobject-detection+4