Mutual Contrastive Learning for Visual Representation Learning
We present a collaborative learning method called Mutual Contrastive Learning (MCL) for general visual representation learning. The core idea of MCL is to perform mutual interaction and transfer of contrastive distributions among a cohort of networks. A crucial component of MCL is Interactive Contrastive Learning (ICL). Compared with vanilla contrastive learning, ICL can aggregate cross-network embedding information and maximize the lower bound to the mutual information between two networks. This enables each network to learn extra contrastive knowledge from others, leading to better feature representations for visual recognition tasks. We emphasize that the resulting MCL is conceptually simple yet empirically powerful. It is a generic framework that can be applied to both supervised and self-supervised representation learning. Experimental results on image classification and transfer learning to object detection show that MCL can lead to consistent performance gains, demonstrating that MCL can guide the network to generate better feature representations. Code is available at https://github.com/winycg/MCL.
Code (1)
Tasks
Contrastive LearningFew-Shot Learningimage-classificationImage ClassificationNetwork Embeddingobject-detectionObject DetectionRepresentation LearningSelf-Supervised Image ClassificationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition
The teacher-free online Knowledge Distillation (KD) aims to train an ensemble of multiple student models collaboratively and distill knowledge from each other. Although existing online KD methods achieve desirable perfor…
Contrastive Learningimage-classificationImage ClassificationKnowledge Distillation+3Conditional Negative Sampling for Contrastive Learning of Visual Representations
Recent methods for learning unsupervised visual representations, dubbed contrastive learning, optimize the noise-contrastive estimation (NCE) bound on mutual information between two views of an image. NCE uses randomly s…
Contrastive LearningInstance SegmentationKeypoint DetectionLinear evaluation+3Mutual Information Estimation as a Difference of Entropies for Unsupervised Representation Learning
Contrastive loss has been successfully exploited in the latest visual unsupervised representation learning methods. Contrastive loss is based on a lower-bound estimation of mutual information where its known limitations …
Mutual Information EstimationRepresentation LearningOn Mutual Information in Contrastive Learning for Visual Representations
In recent years, several unsupervised, "contrastive" learning algorithms in vision have been shown to learn representations that perform remarkably well on transfer tasks. We show that this family of algorithms maximizes…
Contrastive LearningGeneral ClassificationInstance SegmentationKeypoint Detection+3Active Contrastive Learning of Audio-Visual Video Representations
Contrastive learning has been shown to produce generalizable representations of audio and visual data by maximizing the lower bound on the mutual information (MI) between different views of an instance. However, obtainin…
Contrastive LearningRepresentation LearningVideo Classification