A Meta Approach to Defend Noisy Labels by the Manifold Regularizer PSDR
Noisy labels are ubiquitous in real-world datasets, which poses a challenge for robustly training deep neural networks (DNNs) since DNNs can easily overfit to the noisy labels. Most recent efforts have been devoted to defending noisy labels by discarding noisy samples from the training set or assigning weights to training samples, where the weight associated with a noisy sample is expected to be small. Thereby, these previous efforts result in a waste of samples, especially those assigned with small weights. The input $x$ is always useful regardless of whether its observed label $y$ is clean. To make full use of all samples, we introduce a manifold regularizer, named as Paired Softmax Divergence Regularization (PSDR), to penalize the Kullback-Leibler (KL) divergence between softmax outputs of similar inputs. In particular, similar inputs can be effectively generated by data augmentation. PSDR can be easily implemented on any type of DNNs to improve the robustness against noisy labels. As empirically demonstrated on benchmark datasets, our PSDR impressively improve state-of-the-art results by a significant margin.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Defending Backdoor Data Poisoning Attacks by Using Noisy Label Defense Algorithm
Training deep neural networks with data corruption is a challenging problem. One example of such corruption is the backdoor data poisoning attack, in which an adversary strategically injects a backdoor trigger to a small…
Backdoor AttackData PoisoningBatMan-CLR: Making Few-shots Meta-Learners Resilient Against Label Noise
The negative impact of label noise is well studied in classical supervised learning yet remains an open research question in meta-learning. Meta-learners aim to adapt to unseen learning tasks by learning a good initial m…
Meta-LearningScalable low dimensional manifold model in the reconstruction of noisy and incomplete hyperspectral images
We present a scalable low dimensional manifold model for the reconstruction of noisy and incomplete hyperspectral images. The model is based on the observation that the spatial-spectral blocks of a hyperspectral image ty…
Deep Image Prototype Learning with Geometric Heat-Kernel Priors
Learning unsupervised representations of medical imaging cohorts can reveal anatomically meaningful prototypes without expert labels, which are often noisy and fail to capture true pathological heterogeneity. However, ex…
Ensemble Learning with Manifold-Based Data Splitting for Noisy Label Correction
Label noise in training data can significantly degrade a model's generalization performance for supervised learning tasks. Here we focus on the problem that noisy labels are primarily mislabeled samples, which tend to be…
DiversityEnsemble Learning