Pre-train to Gain: Robust Learning Without Clean Labels
Training deep networks with noisy labels leads to poor generalization and degraded accuracy due to overfitting to label noise. Existing approaches for learning with noisy labels often rely on the availability of a clean subset of data. By pre-training a feature extractor backbone without labels using self-supervised learning (SSL), followed by standard supervised training on the noisy dataset, we can train a more noise robust model without requiring a subset with clean labels. We evaluate the use of SimCLR and Barlow~Twins as SSL methods on CIFAR-10 and CIFAR-100 under synthetic and real world noise. Across all noise rates, self-supervised pre-training consistently improves classification accuracy and enhances downstream label-error detection (F1 and Balanced Accuracy). The performance gap widens as the noise rate increases, demonstrating improved robustness. Notably, our approach achieves comparable results to ImageNet pre-trained models at low noise levels, while substantially outperforming them under high noise conditions.
Code (0)
등록된 구현이 없습니다.
Tasks
Learning with noisy labelsSelf-Supervised LearningSimilar Papers 제목 키워드 기반
Low-Loss Subspace Compression for Clean Gains against Multi-Agent Backdoor Attacks
Recent exploration of the multi-agent backdoor attack demonstrated the backfiring effect, a natural defense against backdoor attacks where backdoored inputs are randomly classified. This yields a side-effect of low accur…
Backdoor AttackBootstrapping the Relationship Between Images and Their Clean and Noisy Labels
Many state-of-the-art noisy-label learning methods rely on learning mechanisms that estimate the samples' clean labels during training and discard their original noisy labels. However, this approach prevents the learning…
Image ClassificationLearning with noisy labelsDo We Really Need Gold Samples for Sample Weighting Under Label Noise?
Learning with labels noise has gained significant traction recently due to the sensitivity of deep neural networks under label noise under common loss functions. Losses that are theoretically robust to label noise, howev…
Meta-LearningBackdoor Cleansing with Unlabeled Data
Due to the increasing computational demand of Deep Neural Networks (DNNs), companies and organizations have begun to outsource the training process. However, the externally trained DNNs can potentially be backdoor attack…
Knowledge DistillationImage recognition from raw labels collected without annotators
Image classification problems are typically addressed by first collecting examples with candidate labels, second cleaning the candidate labels manually, and third training a deep neural network on the clean examples. The…
image-classificationImage Classification