Interpolation-based Contrastive Learning for Few-Label Semi-Supervised Learning
Semi-supervised learning (SSL) has long been proved to be an effective technique to construct powerful models with limited labels. In the existing literature, consistency regularization-based methods, which force the perturbed samples to have similar predictions with the original ones have attracted much attention for their promising accuracy. However, we observe that, the performance of such methods decreases drastically when the labels get extremely limited, e.g., 2 or 3 labels for each category. Our empirical study finds that the main problem lies with the drifting of semantic information in the procedure of data augmentation. The problem can be alleviated when enough supervision is provided. However, when little guidance is available, the incorrect regularization would mislead the network and undermine the performance of the algorithm. To tackle the problem, we (1) propose an interpolation-based method to construct more reliable positive sample pairs; (2) design a novel contrastive loss to guide the embedding of the learned network to change linearly between samples so as to improve the discriminative capability of the network by enlarging the margin decision boundaries. Since no destructive regularization is introduced, the performance of our proposed algorithm is largely improved. Specifically, the proposed algorithm outperforms the second best algorithm (Comatch) with 5.3% by achieving 88.73% classification accuracy when only two labels are available for each class on the CIFAR-10 dataset. Moreover, we further prove the generality of the proposed method by improving the performance of the existing state-of-the-art algorithms considerably with our proposed strategy.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningData AugmentationSimilar Papers 제목 키워드 기반
Mixed Graph Contrastive Network for Semi-Supervised Node Classification
Graph Neural Networks (GNNs) have achieved promising performance in semi-supervised node classification in recent years. However, the problem of insufficient supervision, together with representation collapse, largely li…
ClassificationContrastive LearningData AugmentationGraph Learning+2Multi-Objective Interpolation Training for Robustness to Label Noise
Deep neural networks trained with standard cross-entropy loss memorize noisy labels, which degrades their performance. Most research to mitigate this memorization proposes new robust classification loss functions. Conver…
Contrastive LearningImage ClassificationMemorizationRepresentation Learning+1CoDiM: Learning with Noisy Labels via Contrastive Semi-Supervised Learning
Labels are costly and sometimes unreliable. Noisy label learning, semi-supervised learning, and contrastive learning are three different strategies for designing learning processes requiring less annotation cost. Semi-su…
Contrastive LearningImage ClassificationLearning with noisy labelsSemi-Supervised Relational Contrastive Learning
Disease diagnosis from medical images via supervised learning is usually dependent on tedious, error-prone, and costly image labeling by medical experts. Alternatively, semi-supervised learning and self-supervised learni…
Contrastive LearningDiagnosticLesion ClassificationSelf-Supervised Learning+1Rethinking Weak Supervision in Helping Contrastive Learning
Contrastive learning has shown outstanding performances in both supervised and unsupervised learning, and has recently been introduced to solve weakly supervised learning problems such as semi-supervised learning and noi…
Contrastive LearningDenoisingWeakly-supervised Learning