About contrastive unsupervised representation learning for classification and its convergence
Contrastive representation learning has been recently proved to be very efficient for self-supervised training. These methods have been successfully used to train encoders which perform comparably to supervised training on downstream classification tasks. A few works have started to build a theoretical framework around contrastive learning in which guarantees for its performance can be proven. We provide extensions of these results to training with multiple negative samples and for multiway classification. Furthermore, we provide convergence guarantees for the minimization of the contrastive training error with gradient descent of an overparametrized deep neural encoder, and provide some numerical experiments that complement our theoretical findings
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationContrastive LearningGeneral ClassificationRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Convergence of End-to-End Training in Deep Unsupervised Contrastive Learning
Unsupervised contrastive learning has gained increasing attention in the latest research and has proven to be a powerful method for learning representations from unlabeled data. However, little theoretical analysis was k…
Contrastive LearningRanking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives
This paper introduces Ranking Info Noise Contrastive Estimation (RINCE), a new member in the family of InfoNCE losses that preserves a ranked ordering of positive samples. In contrast to the standard InfoNCE loss, which …
Contrastive LearningOut-of-Distribution DetectionRepresentation LearningRetrievalContraCluster: Learning to Classify without Labels by Contrastive Self-Supervision and Prototype-Based Semi-Supervision
The recent advances in representation learning inspire us to take on the challenging problem of unsupervised image classification tasks in a principled way. We propose ContraCluster, an unsupervised image classification …
Contrastive Learningimage-classificationImage ClassificationRepresentation Learning+2Max-Margin Contrastive Learning
Standard contrastive learning approaches usually require a large number of negatives for effective unsupervised learning and often exhibit slow convergence. We suspect this behavior is due to the suboptimal selection of …
Contrastive LearningRepresentation LearningSelf-Supervised Image ClassificationNeural Contrastive Clustering: Fully Unsupervised Bias Reduction for Sentiment Classification
Background: Neural networks produce biased classification results due to correlation bias (they learn correlations between their inputs and outputs to classify samples, even when those correlations do not represent cause…
ClassificationClusteringSentiment AnalysisSentiment Classification+2