Pseudo-label Correction for Instance-dependent Noise Using Teacher-student Framework
The high capacity of deep learning models to learn complex patterns poses a significant challenge when confronted with label noise. The inability to differentiate clean and noisy labels ultimately results in poor generalization. We approach this problem by reassigning the label for each image using a new teacher-student based framework termed P-LC (pseudo-label correction). Traditional teacher-student networks are composed of teacher and student classifiers for knowledge distillation. In our novel approach, we reconfigure the teacher network into a triple encoder, leveraging the triplet loss to establish a pseudo-label correction system. As the student generates pseudo labels for a set of given images, the teacher learns to choose between the initially assigned labels and the pseudo labels. Experiments on MNIST, Fashion-MNIST, and SVHN demonstrate P-LC's superior performance over existing state-of-the-art methods across all noise levels, most notably in high noise. In addition, we introduce a noise level estimation to help assess model performance and inform the need for additional data cleaning procedures.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationPseudo LabelTripletMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Pseudo-Label Quality Decoupling and Correction for Semi-Supervised Instance Segmentation
Semi-Supervised Instance Segmentation (SSIS) involves classifying and grouping image pixels into distinct object instances using limited labeled data. This learning paradigm usually faces a significant challenge of unsta…
Instance SegmentationPseudo LabelSemantic SegmentationSemi-Supervised Instance SegmentationAn Instance-Dependent Simulation Framework for Learning with Label Noise
We propose a simulation framework for generating instance-dependent noisy labels via a pseudo-labeling paradigm. We show that the distribution of the synthetic noisy labels generated with our framework is closer to human…
Learning with noisy labelsConfidence Scores Make Instance-dependent Label-noise Learning Possible
In learning with noisy labels, for every instance, its label can randomly walk to other classes following a transition distribution which is named a noise model. Well-studied noise models are all instance-independent, na…
Learning with noisy labelsTwo Wrongs Don't Make a Right: Combating Confirmation Bias in Learning with Label Noise
Noisy labels damage the performance of deep networks. For robust learning, a prominent two-stage pipeline alternates between eliminating possible incorrect labels and semi-supervised training. However, discarding part of…
Image ClassificationMitigating Instance-Dependent Label Noise: Integrating Self-Supervised Pretraining with Pseudo-Label Refinement
Deep learning models rely heavily on large volumes of labeled data to achieve high performance. However, real-world datasets often contain noisy labels due to human error, ambiguity, or resource constraints during the an…
Pseudo LabelSelf-Supervised Learning