paper-with-me

Papers

Word-Level Uncertainty Estimation for Black-Box Text Classifiers using RNNs

2020-12-01 · COLING 2020 8 · Jakob Smedegaard Andersen, Tom Sch{\"o}ner, Walid Maalej

Estimating uncertainties of Neural Network predictions paves the way towards more reliable and trustful text classifications. However, common uncertainty estimation approaches remain as black-boxes without explaining which features have led to the uncertainty of a prediction. This hinders users from understanding the cause of unreliable model behaviour. We introduce an approach to decompose and visualize the uncertainty of text classifiers at the level of words. Our approach builds on top of Recurrent Neural Networks and Bayesian modelling in order to provide detailed explanations of uncertainties, enabling a deeper reasoning about unreliable model behaviours. We conduct a preliminary experiment to check the impact and correctness of our approach. By explaining and investigating the predictive uncertainties of a sentiment analysis task, we argue that our approach is able to provide a more profound understanding of artificial decision making.

📄 PDF Abstract BibTeX

Code (1)

jsandersen/wu-rnn 공식 구현 tf

Tasks

Decision MakingSentiment Analysis

Similar Papers 제목 키워드 기반

Font Identification in Historical Documents Using Active Learning

2016-01-27 · Anshul Gupta, Ricardo Gutierrez-Osuna, Matthew Christy, Richard Furuta 외

Identifying the type of font (e.g., Roman, Blackletter) used in historical documents can help optical character recognition (OCR) systems produce more accurate text transcriptions. Towards this end, we present an active-…

Active LearningDiversityOptical Character RecognitionOptical Character Recognition (OCR)

BiG-SURE - Bipartite Graph for Semantic Uncertainty and Reliability Estimation of LLMs

2026-08-31 · Debarpan Bhattacharya, Malay Phadke, Sriram Ganapathy arxiv

Reliable uncertainty estimation is a crucial requirement for deploying large language models (LLMs) and vision-language models (VLMs) in safety-critical settings, especially when the model parameters are not accessible (…

Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach

2024-04-24 · Linyu Liu, Yu Pan, Xiaocheng Li, Guanting Chen

In this paper, we study the problem of uncertainty estimation and calibration for LLMs. We begin by formulating the uncertainty estimation problem, a relevant yet underexplored area in existing literature. We then propos…

Investigating the Helpfulness of Word-Level Quality Estimation for Post-Editing Machine Translation Output

2021-11-01 · EMNLP 2021 11 · Raksha Shenoy, Nico Herbig, Antonio Krüger, Josef van Genabith

Compared to fully manual translation, post-editing (PE) machine translation (MT) output can save time and reduce errors. Automatic word-level quality estimation (QE) aims to predict the correctness of words in MT output …

Machine TranslationTranslation

Uncertainty in Natural Language Processing: Sources, Quantification, and Applications

2023-06-05 · Mengting Hu, Zhen Zhang, Shiwan Zhao, Minlie Huang 외

As a main field of artificial intelligence, natural language processing (NLP) has achieved remarkable success via deep neural networks. Plenty of NLP tasks have been addressed in a unified manner, with various tasks bein…

Uncertainty Quantification