Uncertainty Estimation on Sequential Labeling via Uncertainty Transmission
Sequential labeling is a task predicting labels for each token in a sequence, such as Named Entity Recognition (NER). NER tasks aim to extract entities and predict their labels given a text, which is important in information extraction. Although previous works have shown great progress in improving NER performance, uncertainty estimation on NER (UE-NER) is still underexplored but essential. This work focuses on UE-NER, which aims to estimate uncertainty scores for the NER predictions. Previous uncertainty estimation models often overlook two unique characteristics of NER: the connection between entities (i.e., one entity embedding is learned based on the other ones) and wrong span cases in the entity extraction subtask. Therefore, we propose a Sequential Labeling Posterior Network (SLPN) to estimate uncertainty scores for the extracted entities, considering uncertainty transmitted from other tokens. Moreover, we have defined an evaluation strategy to address the specificity of wrong-span cases. Our SLPN has achieved significant improvements on three datasets, such as a 5.54-point improvement in AUPR on the MIT-Restaurant dataset. Our code is available at \url{https://github.com/he159ok/UncSeqLabeling_SLPN}.
Code (1)
Tasks
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NERSpecificitySimilar Papers 제목 키워드 기반
Transmission-Guided Bayesian Generative Model for Smoke Segmentation
Smoke segmentation is essential to precisely localize wildfire so that it can be extinguished in an early phase. Although deep neural networks have achieved promising results on image segmentation tasks, they are prone t…
Image DehazingImage SegmentationmodelSegmentation+1Leveraging Labeling Representations in Uncertainty-based Semi-supervised Segmentation
Semi-supervised segmentation tackles the scarcity of annotations by leveraging unlabeled data with a small amount of labeled data. A prominent way to utilize the unlabeled data is by consistency training which commonly u…
SegmentationIncremental Uncertainty-aware Performance Monitoring with Active Labeling Intervention
We study the problem of monitoring machine learning models under gradual distribution shifts, where circumstances change slowly over time, often leading to unnoticed yet significant declines in accuracy. To address this,…
Evaluating Prediction Uncertainty Estimates from BatchEnsemble
Deep learning models struggle with uncertainty estimation. Many approaches are either computationally infeasible or underestimate uncertainty. We investigate \textit{BatchEnsemble} as a general and scalable method for un…
On uncertainty estimation in active learning for image segmentation
Uncertainty estimation is important for interpreting the trustworthiness of machine learning models in many applications. This is especially critical in the data-driven active learning setting where the goal is to achiev…
Active LearningImage SegmentationMedical Image SegmentationSemantic Segmentation