Mapping from Meaning: Addressing the Miscalibration of Prompt-Sensitive Language Models
An interesting behavior in large language models (LLMs) is prompt sensitivity. When provided with different but semantically equivalent versions of the same prompt, models may produce very different distributions of answers. This suggests that the uncertainty reflected in a model's output distribution for one prompt may not reflect the model's uncertainty about the meaning of the prompt. We model prompt sensitivity as a type of generalization error, and show that sampling across the semantic ``concept space'' with paraphrasing perturbations improves uncertainty calibration without compromising accuracy. Additionally, we introduce a new metric for uncertainty decomposition in black-box LLMs that improves upon entropy-based decomposition by modeling semantic continuities in natural language generation. We show that this decomposition metric can be used to quantify how much LLM uncertainty is attributed to prompt sensitivity. Our work introduces a new way to improve uncertainty calibration in prompt-sensitive language models, and provides evidence that some LLMs fail to exhibit consistent general reasoning about the meanings of their inputs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Probability calibration for precipitation nowcasting
Reliable precipitation nowcasting is critical for weather-sensitive decision-making, yet neural weather models (NWMs) can produce poorly calibrated probabilistic forecasts. Standard calibration metrics such as the expect…
Calibrate to Discriminate: Improve In-Context Learning with Label-Free Comparative Inference
While in-context learning with large language models (LLMs) has shown impressive performance, we have discovered a unique miscalibration behavior where both correct and incorrect predictions are assigned the same level o…
In-Context LearningA Study on the Calibration of In-context Learning
Accurate uncertainty quantification is crucial for the safe deployment of machine learning models, and prior research has demonstrated improvements in the calibration of modern language models (LMs). We study in-context …
In-Context LearningNatural Language UnderstandingUncertainty QuantificationPriors Persist Through Suppression: A Stroop Paradigm for Lexical Override
Glossaries, technical specifications, and system prompts routinely ask language models to use familiar words in unfamiliar ways. The instruction competes with what the word already means, and even when it wins, the pretr…
On the Entropy Calibration of Language Models
We study the problem of entropy calibration, which asks whether a language model's entropy over generations matches its log loss on human text. Past work found that models are miscalibrated, with entropy per step increas…