Logistic-Normal Likelihoods for Heteroscedastic Label Noise
A natural way of estimating heteroscedastic label noise in regression is to model the observed (potentially noisy) target as a sample from a normal distribution, whose parameters can be learned by minimizing the negative log-likelihood. This formulation has desirable loss attenuation properties, as it reduces the contribution of high-error examples. Intuitively, this behavior can improve robustness against label noise by reducing overfitting. We propose an extension of this simple and probabilistic approach to classification that has the same desirable loss attenuation properties. Furthermore, we discuss and address some practical challenges of this extension. We evaluate the effectiveness of the method by measuring its robustness against label noise in classification. We perform enlightening experiments exploring the inner workings of the method, including sensitivity to hyperparameters, ablation studies, and other insightful analyses.
Code (1)
Tasks
ClassificationSimilar Papers 제목 키워드 기반
Variational Variance: Simple, Reliable, Calibrated Heteroscedastic Noise Variance Parameterization
Brittle optimization has been observed to adversely impact model likelihoods for regression and VAEs when simultaneously fitting neural network mappings from a (random) variable onto the mean and variance of a dependent …
regressionActive Heteroscedastic Regression
An active learner is given a model class $\Theta$, a large sample of unlabeled data drawn from an underlying distribution and access to a labeling oracle that can provide a label for any of the unlabeled instances. …
Active LearningBinary ClassificationregressionNonparametric Deconvolution and Denoising using Simulation Based Inference
Latent signals are often obscured by measurement noise, yet encode the underlying laws and dynamics of complex systems; learning both the signals and their distributions remains a central challenge in scientific inferenc…
A Simple Probabilistic Method for Deep Classification under Input-Dependent Label Noise
Datasets with noisy labels are a common occurrence in practical applications of classification methods. We propose a simple probabilistic method for training deep classifiers under input-dependent (heteroscedastic) label…
General Classificationimage-classificationImage ClassificationImage Segmentation+1Correlated Input-Dependent Label Noise in Large-Scale Image Classification
Large scale image classification datasets often contain noisy labels. We take a principled probabilistic approach to modelling input-dependent, also known as heteroscedastic, label noise in these datasets. We place a mul…
Classificationimage-classificationImage Classification