The Bad Batches: Enhancing Self-Supervised Learning in Image Classification Through Representative Batch Curation
The pursuit of learning robust representations without human supervision is a longstanding challenge. The recent advancements in self-supervised contrastive learning approaches have demonstrated high performance across various representation learning challenges. However, current methods depend on the random transformation of training examples, resulting in some cases of unrepresentative positive pairs that can have a large impact on learning. This limitation not only impedes the convergence of the learning process but the robustness of the learnt representation as well as requiring larger batch sizes to improve robustness to such bad batches. This paper attempts to alleviate the influence of false positive and false negative pairs by employing pairwise similarity calculations through the Fr\'echet ResNet Distance (FRD), thereby obtaining robust representations from unlabelled data. The effectiveness of the proposed method is substantiated by empirical results, where a linear classifier trained on self-supervised contrastive representations achieved an impressive 87.74\% top-1 accuracy on STL10 and 99.31\% on the Flower102 dataset. These results emphasize the potential of the proposed approach in pushing the boundaries of the state-of-the-art in self-supervised contrastive learning, particularly for image classification tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive Learningimage-classificationImage ClassificationRepresentation LearningSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Jigsaw Clustering for Unsupervised Visual Representation Learning
Unsupervised representation learning with contrastive learning achieved great success. This line of methods duplicate each training batch to construct contrastive pairs, making each training batch and its augmented versi…
ClusteringContrastive LearningRepresentation LearningMoBYv2AL: Self-supervised Active Learning for Image Classification
Active learning(AL) has recently gained popularity for deep learning(DL) models. This is due to efficient and informative sampling, especially when the learner requires large-scale labelled datasets. Commonly, the sampli…
Active LearningClassificationimage-classificationImage Classification+1Investigating a Baseline Of Self Supervised Learning Towards Reducing Labeling Costs For Image Classification
Data labeling in supervised learning is considered an expensive and infeasible tool in some conditions. The self-supervised learning method is proposed to tackle the learning effectiveness with fewer labeled data, howeve…
image-classificationImage ClassificationSelf-Supervised LearningImbalance-Aware Self-Supervised Learning for 3D Radiomic Representations
Radiomic representations can quantify properties of regions of interest in medical image data. Classically, they account for pre-defined statistics of shape, texture, and other low-level image features. Alternatively, de…
Brain Tumor ClassificationRepresentation LearningSelf-Supervised LearningAdversarial Momentum-Contrastive Pre-Training
Recently proposed adversarial self-supervised learning methods usually require big batches and long training epochs to extract robust features, which will bring heavy computational overhead on platforms with limited reso…
Contrastive LearningData AugmentationSelf-Supervised Learning