Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinations

2024-04-16 · Christian Tomani, Kamalika Chaudhuri, Ivan Evtimov, Daniel Cremers, Mark Ibrahim

A major barrier towards the practical deployment of large language models (LLMs) is their lack of reliability. Three situations where this is particularly apparent are correctness, hallucinations when given unanswerable questions, and safety. In all three cases, models should ideally abstain from responding, much like humans, whose ability to understand uncertainty makes us refrain from answering questions we don't know. Inspired by analogous approaches in classification, this study explores the feasibility and efficacy of abstaining while uncertain in the context of LLMs within the domain of question-answering. We investigate two kinds of uncertainties, statistical uncertainty metrics and a distinct verbalized measure, termed as In-Dialogue Uncertainty (InDU). Using these uncertainty measures combined with models with and without Reinforcement Learning with Human Feedback (RLHF), we show that in all three situations, abstention based on the right kind of uncertainty measure can boost the reliability of LLMs. By sacrificing only a few highly uncertain samples we can improve correctness by 2% to 8%, avoid 50% hallucinations via correctly identifying unanswerable questions and increase safety by 70% up to 99% with almost no additional computational overhead.

📄 PDF Abstract BibTeX arXiv:2404.10960

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering

Knowing When to Abstain: Medical LLMs Under Clinical Uncertainty

2026-01-18 · Sravanthi Machcha, Sushrita Yerra, Sahil Gupta, Aishwarya Sahoo 외 arxiv

Current evaluation of large language models (LLMs) overwhelmingly prioritizes accuracy; however, in real-world and safety-critical applications, the ability to abstain when uncertain is equally vital for trustworthy depl…

Question Answering

Rethinking LLM Verification: Evidence Structure, Uncertainty, and Selective Refinement

2026-08-11 · Uma Ranjan, Kunal Tilaganji, Aditya Koul, Anurag Mahipal 외 arxiv

Large language models (LLMs) often rely on shortcuts rather than systematic reasoning, raising safety concerns in medical applications. Allowing models to abstain when uncertain improves reliability but introduces a cove…

When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?

2026-02-04 · Xinyu Zhou, Chang Jin, Carsten Eickhoff, Zhijiang Guo 외 arxiv

Large language models (LLMs) rarely admit uncertainty, often producing fluent but misleading answers, rather than abstaining (i.e., refusing to answer). This weakness is even evident in temporal question answering, where…

Reinforcement Learning Question Answering Knowledge Graphs

Learning Conformal Abstention Policies for Adaptive Risk Management in Large Language and Vision-Language Models

2025-02-08 · Sina Tayebati, Divake Kumar, Nastaran Darabi, Dinithi Jayasuriya 외

Large Language and Vision-Language Models (LLMs/VLMs) are increasingly used in safety-critical applications, yet their opaque decision-making complicates risk assessment and reliability. Uncertainty quantification (UQ) h…

Conformal Prediction Decision Making Hallucination Informativeness+4

TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning

2025-09-30 · Zhepei Wei, Xiao Yang, Kai Sun, Jiaqi Wang 외 arxiv

While large language models (LLMs) have demonstrated strong performance on factoid question answering, they are still prone to hallucination and untruthful responses, particularly when tasks demand information outside th…

General Reinforcement Learning Question Answering