Understanding and Utilizing Deep Neural Networks Trained with Noisy Labels
Noisy labels are ubiquitous in real-world datasets, which poses a challenge for robustly training deep neural networks (DNNs) as DNNs usually have the high capacity to memorize the noisy labels. In this paper, we find that the test accuracy can be quantitatively characterized in terms of the noise ratio in datasets. In particular, the test accuracy is a quadratic function of the noise ratio in the case of symmetric noise, which explains the experimental findings previously published. Based on our analysis, we apply cross-validation to randomly split noisy datasets, which identifies most samples that have correct labels. Then we adopt the Co-teaching strategy which takes full advantage of the identified samples to train DNNs robustly against noisy labels. Compared with extensive state-of-the-art methods, our strategy consistently improves the generalization performance of DNNs under both synthetic and real-world training noise.
Code (3)
Tasks
Image ClassificationSimilar Papers 제목 키워드 기반
Self-refining of Pseudo Labels for Music Source Separation with Noisy Labeled Data
Music source separation (MSS) faces challenges due to the limited availability of correctly-labeled individual instrument tracks. With the push to acquire larger datasets to improve MSS performance, the inevitability of …
Instrument RecognitionMusic Source SeparationKnowledge Distillation with Noisy Labels for Natural Language Understanding
Knowledge Distillation (KD) is extensively used to compress and deploy large pre-trained language models on edge devices for real-world applications. However, one neglected area of research is the impact of noisy (corrup…
Knowledge DistillationNatural Language UnderstandingOn the Robustness of Monte Carlo Dropout Trained with Noisy Labels
The memorization effect of deep learning hinders its performance to effectively generalize on test set when learning with noisy labels. Prior study has discovered that epistemic uncertainty techniques are robust when tra…
Learning with noisy labelsMemorizationDimensionality-Driven Learning with Noisy Labels
Datasets with significant proportions of noisy (incorrect) class labels present challenges for training accurate Deep Neural Networks (DNNs). We propose a new perspective for understanding DNN generalization for such dat…
Image ClassificationLearning with noisy labelsDeep Neural Networks Learn Meta-Structures from Noisy Labels in Semantic Segmentation
How deep neural networks (DNNs) learn from noisy labels has been studied extensively in image classification but much less in image segmentation. So far, our understanding of the learning behavior of DNNs trained by nois…
image-classificationImage ClassificationImage SegmentationSegmentation+1