Noisy Label Regularisation for Textual Regression
Training with noisy labelled data is known to be detrimental to model performance, especially for high-capacity neural network models in low-resource domains. Our experiments suggest that standard regularisation strategies, such as weight decay and dropout, are ineffective in the face of noisy labels. We propose a simple noisy label detection method that prevents error propagation from the input layer. The approach is based on the observation that the projection of noisy labels is learned through memorisation at advanced stages of learning, and that the Pearson correlation is sensitive to outliers. Extensive experiments over real-world human-disagreement annotations as well as randomly-corrupted and data-augmented labels, across various tasks and domains, demonstrate that our method is effective, regularising noisy labels and improving generalisation performance.
Code (1)
Tasks
regressionSimilar Papers 제목 키워드 기반
Rates of Convergence for Regression with the Graph Poly-Laplacian
In the (special) smoothing spline problem one considers a variational problem with a quadratic data fidelity penalty and Laplacian regularisation. Higher order regularity can be obtained via replacing the Laplacian regul…
regressionOut-of-distribution robustness for multivariate analysis via causal regularisation
We propose a regularisation strategy of classical machine learning algorithms rooted in causality that ensures robustness against distribution shifts. Building upon the anchor regression framework, we demonstrate how inc…
Causal InferenceregressionRobust One-Class Kernel Spectral Regression
The kernel null-space technique and its regression-based formulation (called one-class kernel spectral regression, a.k.a. OC-KSR) is known to be an effective and computationally attractive one-class classification framew…
General ClassificationOne-Class ClassificationregressionUPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression
Noisy self-reported empathy scores challenge supervised learning for empathy regression. While many algorithms have been proposed for learning with noisy labels in textual classification problems, the regression counterp…
Learning with noisy labelsLanguage ModellingComparing regularisation paths of (conjugate) gradient estimators in ridge regression
We consider standard gradient descent, gradient flow and conjugate gradients as iterative algorithms for minimizing a penalized ridge criterion in linear regression. While it is well known that conjugate gradients exhibi…
regression