Bayesian Self-Supervised Contrastive Learning
Recent years have witnessed many successful applications of contrastive learning in diverse domains, yet its self-supervised version still remains many exciting challenges. As the negative samples are drawn from unlabeled datasets, a randomly selected sample may be actually a false negative to an anchor, leading to incorrect encoder training. This paper proposes a new self-supervised contrastive loss called the BCL loss that still uses random samples from the unlabeled data while correcting the resulting bias with importance weights. The key idea is to design the desired sampling distribution for sampling hard true negative samples under the Bayesian framework. The prominent advantage lies in that the desired sampling distribution is a parametric structure, with a location parameter for debiasing false negative and concentration parameter for mining hard negative, respectively. Experiments validate the effectiveness and superiority of the BCL loss.
Code (1)
Tasks
Contrastive LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Incorporating Unlabelled Data into Bayesian Neural Networks
Conventional Bayesian Neural Networks (BNNs) are unable to leverage unlabelled data to improve their predictions. To overcome this limitation, we introduce Self-Supervised Bayesian Neural Networks, which use unlabelled d…
Active LearningSelf-Supervised LearningUnsupervised Pre-trainingContrastive Learning for Space-Time Correspondence via Self-Cycle Consistency
We propose a novel probabilistic method employing Bayesian Model Averaging and self-cycle regularization for spatio-temporal correspondence learning in videos within a self-supervised learning framework. Most existin…
Contrastive LearningSelf-Supervised LearningBayesian Graph Contrastive Learning
Contrastive learning has become a key component of self-supervised learning approaches for graph-structured data. Despite their success, existing graph contrastive learning methods are incapable of uncertainty quantifica…
Contrastive LearningSelf-Supervised LearningUncertainty QuantificationSelf-Supervised Learning for 3D Medical Image Analysis using 3D SimCLR and Monte Carlo Dropout
Self-supervised learning methods can be used to learn meaningful representations from unlabeled data that can be transferred to supervised downstream tasks to reduce the need for labeled data. In this paper, we propose a…
Brain Tumor SegmentationMedical Image AnalysisMedical Image SegmentationSegmentation+2Contrastive Bayesian Analysis for Deep Metric Learning
Recent methods for deep metric learning have been focusing on designing different contrastive loss functions between positive and negative pairs of samples so that the learned feature embedding is able to pull positive s…
Contrastive LearningMetric Learning