Do We Need to Penalize Variance of Losses for Learning with Label Noise?
Algorithms which minimize the averaged loss have been widely designed for dealing with noisy labels. Intuitively, when there is a finite training sample, penalizing the variance of losses will improve the stability and generalization of the algorithms. Interestingly, we found that the variance should be increased for the problem of learning with noisy labels. Specifically, increasing the variance will boost the memorization effects and reduce the harmfulness of incorrect labels. By exploiting the label noise transition matrix, regularizers can be easily designed to reduce the variance of losses and be plugged in many existing algorithms. Empirically, the proposed method by increasing the variance of losses significantly improves the generalization ability of baselines on both synthetic and real-world datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Learning with noisy labelsMemorizationSimilar Papers 제목 키워드 기반
Label Noise SGD Provably Prefers Flat Global Minimizers
In overparametrized models, the noise in stochastic gradient descent (SGD) implicitly regularizes the optimization trajectory and determines which local minimum SGD converges to. Motivated by empirical studies that demon…
A Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance. To mitigate this issue, we propose a simple but effective method that is …
Learning with noisy labelsLosses over Labels: Weakly Supervised Learning via Direct Loss Construction
Owing to the prohibitive costs of generating large amounts of labeled data, programmatic weak supervision is a growing paradigm within machine learning. In this setting, users design heuristics that provide noisy labels …
feature selectionimage-classificationImage ClassificationWeakly-supervised LearningA Simple yet Effective Baseline for Robust Deep Learning with Noisy Labels
Recently deep neural networks have shown their capacity to memorize training data, even with noisy labels, which hurts generalization performance. To mitigate this issue, we provide a simple but effective baseline method…
Learning with noisy labelsSelf-Filtering: A Noise-Aware Sample Selection for Label Noise with Confidence Penalization
Sample selection is an effective strategy to mitigate the effect of label noise in robust learning. Typical strategies commonly apply the small-loss criterion to identify clean samples. However, those samples lying aroun…
Learning with noisy labels