Handling Realistic Label Noise in BERT Text Classification
Labels noise refers to errors in training labels caused by cheap data annotation methods, such as web scraping or crowd-sourcing, which can be detrimental to the performance of supervised classifiers. Several methods have been proposed to counteract the effect of random label noise in supervised classification, and some studies have shown that BERT is already robust against high rates of randomly injected label noise. However, real label noise is not random; rather, it is often correlated with input features or other annotator-specific factors. In this paper, we evaluate BERT in the presence of two types of realistic label noise: feature-dependent label noise, and synthetic label noise from annotator disagreements. We show that the presence of these types of noise significantly degrades BERT classification performance. To improve robustness, we evaluate different types of ensembles and noise-cleaning methods and compare their effectiveness against label noise across different datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Classificationtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Is BERT Robust to Label Noise? A Study on Learning with Noisy Labels in Text Classification
Incorrect labels in training data occur when human annotators make mistakes or when the data is generated via weak or distant supervision. It has been shown that complex noise-handling techniques - by modeling, cleaning …
Learning with noisy labelstext-classificationText ClassificationTask-Adaptive Pre-Training for Boosting Learning With Noisy Labels: A Study on Text Classification for African Languages
For high-resource languages like English, text classification is a well-studied task. The performance of modern NLP models easily achieves an accuracy of more than 90% in many standard datasets for text classification in…
ClassificationLearning with noisy labelstext-classificationText ClassificationDelving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmark
Graph Neural Networks (GNNs) have achieved state-of-the-art performance in node classification tasks but struggle with label noise in real-world data. Existing studies on graph learning with label noise commonly rely on …
BenchmarkingGraph LearningNode ClassificationInstance-dependent Noisy-label Learning with Graphical Model Based Noise-rate Estimation
Deep learning faces a formidable challenge when handling noisy labels, as models tend to overfit samples affected by label noise. This challenge is further compounded by the presence of instance-dependent noise (IDN), a …
CG-BERT: Conditional Text Generation with BERT for Generalized Few-shot Intent Detection
In this paper, we formulate a more realistic and difficult problem setup for the intent detection task in natural language understanding, namely Generalized Few-Shot Intent Detection (GFSID). GFSID aims to discriminate a…
Conditional Text GenerationIntent DetectionLanguage ModelingLanguage Modelling+3