Let the Model Distribute Its Doubt: Confidence Estimation through Verbalized Probability Distribution
Knowing the reliability of a model's response is essential in practical applications. Given the strong generation capabilities of large language models (LLMs), research has focused on generating verbalized confidence. This approach is further enhanced by integrating chain-of-thought reasoning, which provides logical and transparent estimates. However, how reasoning strategies affect the estimated confidence remains under-explored. In this work, we demonstrate that predicting a verbalized probability distribution effectively promotes reasoning for confidence estimation. It requires an LLM to consider all possible answers rather than relying on a single guess, and the requirement of producing a distribution elicits more careful confidence assignment. We conduct systematic experiments comparing different verbalization-based methods across multiple LLMs and tasks. Our method consistently shows advantages, whether in the simple prompting setup or after optimization via reinforcement learning (RL). Notably, it achieves higher reasoning efficacy during inference-time scaling, saving nearly 6$\times$ the computation to reach the best Brier score of the strongest baseline on MMLU-Pro. Additionally, we reveal its limitations on specific tasks and discuss possible solutions for broader applicability.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratio
Uncertainty estimation for reasoning language models remains difficult to deploy in practice: sampling-based methods are computationally expensive, while common single-pass proxies such as verbalized confidence or trace …
Reducing conversational agents' overconfidence through linguistic calibration
While improving neural dialogue agents' factual accuracy is the object of much research, another important aspect of communication, less studied in the setting of neural dialogue, is transparency about ignorance. In this…
ADVICE: Answer-Dependent Verbalized Confidence Estimation
Recent progress in large language models (LLMs) has enabled them to communicate their confidence in natural language, improving transparency and reliability. However, this expressiveness is often accompanied by systemati…
ORCE: Order-Aware Alignment of Verbalized Confidence in Large Language Models
Large language models (LLMs) often produce answers with high certainty even when they are incorrect, making reliable confidence estimation essential for deployment in real-world scenarios. Verbalized confidence, where mo…
Reinforcement LearningAnswer GenerationReinforcement Learning for Better Verbalized Confidence in Long-Form Generation
Hallucination remains a major challenge for the safe and trustworthy deployment of large language models (LLMs) in factual content generation. Prior work has explored confidence estimation as an effective approach to hal…
FormHallucinationQuestion AnsweringReinforcement Learning (RL)