Bayesian Methods for Semi-supervised Text Annotation
Human annotations are an important source of information in the development of natural language understanding approaches. As under the pressure of productivity annotators can assign different labels to a given text, the quality of produced annotations frequently varies. This is especially the case if decisions are difficult, with high cognitive load, requires awareness of broader context, or careful consideration of background knowledge. To alleviate the problem, we propose two semi-supervised methods to guide the annotation process: a Bayesian deep learning model and a Bayesian ensemble method. Using a Bayesian deep learning method, we can discover annotations that cannot be trusted and might require reannotation. A recently proposed Bayesian ensemble method helps us to combine the annotators' labels with predictions of trained models. According to the results obtained from three hate speech detection experiments, the proposed Bayesian methods can improve the annotations and prediction performance of BERT models.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningHate Speech DetectionNatural Language Understandingtext annotationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bayesian Self-Training for Semi-Supervised 3D Segmentation
3D segmentation is a core problem in computer vision and, similarly to many other dense prediction tasks, it requires large amounts of annotated data for adequate training. However, densely labeling 3D point clouds to em…
3D Instance Segmentation3D Semantic Segmentation3D visual groundingInstance Segmentation+3Combining Minimally-supervised Methods for Arabic Named Entity Recognition
Supervised methods can achieve high performance on NLP tasks, such as Named Entity Recognition (NER), but new annotations are required for every new domain and/or genre change. This has motivated research in minimally su…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+1Semi-Supervised and Unsupervised Sense Annotation via Translations
Acquisition of multilingual training data continues to be a challenge in word sense disambiguation (WSD). To address this problem, unsupervised approaches have been proposed to automatically generate sense annotations fo…
Machine TranslationTranslationWord Sense DisambiguationNonparametric Bayesian Semi-supervised Word Segmentation
This paper presents a novel hybrid generative/discriminative model of word segmentation based on nonparametric Bayesian methods. Unlike ordinary discriminative word segmentation which relies only on labeled data, our sem…
Language ModelingLanguage ModellingMachine TranslationSegmentation+1Making Better Use of Unlabelled Data in Bayesian Active Learning
Fully supervised models are predominant in Bayesian active learning. We argue that their neglect of the information present in unlabelled data harms not just predictive performance but also decisions about what data to a…
Active Learning