Correlated Input-Dependent Label Noise in Large-Scale Image Classification
Large scale image classification datasets often contain noisy labels. We take a principled probabilistic approach to modelling input-dependent, also known as heteroscedastic, label noise in these datasets. We place a multivariate Normal distributed latent variable on the final hidden layer of a neural network classifier. The covariance matrix of this latent variable, models the aleatoric uncertainty due to label noise. We demonstrate that the learned covariance structure captures known sources of label noise between semantically similar and co-occurring classes. Compared to standard neural network training and other baselines, we show significantly improved accuracy on Imagenet ILSVRC 2012 79.3% (+2.6%), Imagenet-21k 47.0% (+1.1%) and JFT 64.7% (+1.6%). We set a new state-of-the-art result on WebVision 1.0 with 76.6% top-1 accuracy. These datasets range from over 1M to over 300M training examples and from 1k classes to more than 21k classes. Our method is simple to use, and we provide an implementation that is a drop-in replacement for the final fully-connected layer in a deep classifier.
Code (0)
등록된 구현이 없습니다.
Tasks
Classificationimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
Handling Realistic Label Noise in BERT Text Classification
Labels noise refers to errors in training labels caused by cheap data annotation methods, such as web scraping or crowd-sourcing, which can be detrimental to the performance of supervised classifiers. Several methods hav…
Classificationtext-classificationText ClassificationUnsupervised Denoising for Signal-Dependent and Row-Correlated Imaging Noise
Accurate analysis of microscopy images is hindered by the presence of noise. This noise is usually signal-dependent and often additionally correlated along rows or columns of pixels. Current self- and unsupervised denois…
DecoderDenoisingImage DenoisingMedian2Median: Zero-shot Suppression of Structured Noise in Images
Image denoising is a fundamental problem in computer vision and medical imaging. However, real-world images are often degraded by structured noise with strong anisotropic correlations that existing methods struggle to re…
Image DenoisingDisentangling Correlated Speaker and Noise for Speech Synthesis via Data Augmentation and Adversarial Factorization
To leverage crowd-sourced data to train multi-speaker text-to-speech (TTS) models that can synthesize clean speech for all speakers, it is essential to learn disentangled representations which can independently control t…
Data AugmentationDisentanglementSpeech Synthesistext-to-speech+1PCA in Data-Dependent Noise (Correlated-PCA): Nearly Optimal Finite Sample Guarantees
We study Principal Component Analysis (PCA) in a setting where a part of the corrupting noise is data-dependent and, as a result, the noise and the true data are correlated. Under a bounded-ness assumption on the true da…