Understanding Instance-Level Label Noise: Disparate Impacts and Treatments
This paper aims to provide understandings for the effect of an over-parameterized model, e.g. a deep neural network, memorizing instance-dependent noisy labels. We first quantify the harms caused by memorizing noisy instances, and show the disparate impacts of noisy labels for sample instances with different representation frequencies. We then analyze how several popular solutions for learning with noisy labels mitigate this harm at the instance level. Our analysis reveals that existing approaches lead to disparate treatments when handling noisy instances. While higher-frequency instances often enjoy a high probability of an improvement by applying these solutions, lower-frequency instances do not. Our analysis reveals new understandings for when these approaches work, and provides theoretical justifications for previously reported empirical observations. This observation requires us to rethink the distribution of label noise across instances and calls for different treatments for instances in different regimes.
Code (0)
등록된 구현이 없습니다.
Tasks
Learning with noisy labelsSimilar Papers 제목 키워드 기반
Identifiability of Label Noise Transition Matrix
The noise transition matrix plays a central role in the problem of learning with noisy labels. Among many other reasons, a large number of existing solutions rely on access to it. Identifying and estimating the transitio…
Learning with noisy labelsDisparate Censorship & Undertesting: A Source of Label Bias in Clinical Machine Learning
As machine learning (ML) models gain traction in clinical applications, understanding the impact of clinician and societal biases on ML models is increasingly important. While biases can arise in the labels used for mode…
BIG-bench Machine LearningDiagnosticTraining Gradient Boosted Decision Trees on Tabular Data Containing Label Noise for Classification Tasks
Label noise, which refers to the mislabeling of instances in a dataset, can significantly impair classifier performance, increase model complexity, and affect feature selection. While most research has concentrated on de…
feature selectionConfidence Scores Make Instance-dependent Label-noise Learning Possible
In learning with noisy labels, for every instance, its label can randomly walk to other classes following a transition distribution which is named a noise model. Well-studied noise models are all instance-independent, na…
Learning with noisy labelsCamoTeacher: Dual-Rotation Consistency Learning for Semi-Supervised Camouflaged Object Detection
Existing camouflaged object detection~(COD) methods depend heavily on large-scale pixel-level annotations.However, acquiring such annotations is laborious due to the inherent camouflage characteristics of the objects.Sem…
object-detectionObject DetectionPseudo Label