Double-Calibration: Towards Reliable LLMs via Calibrating Knowledge and Reasoning Confidence
Reliable reasoning in Large Language Models (LLMs) is challenged by their propensity for hallucination. While augmenting LLMs with Knowledge Graphs (KGs) improves factual accuracy, existing KG-augmented methods fail to quantify epistemic uncertainty in both the retrieved evidence and LLMs' reasoning. To bridge this gap, we introduce DoublyCal, a framework built on a novel double-calibration principle. DoublyCal employs a lightweight proxy model to first generate KG evidence alongside a calibrated evidence confidence. This calibrated supporting evidence then guides a black-box LLM, yielding final predictions that are not only more accurate but also well-calibrated, with confidence scores traceable to the uncertainty of the supporting evidence. Experiments on knowledge-intensive benchmarks show that DoublyCal significantly improves both the accuracy and confidence calibration of black-box LLMs while maintaining low token cost.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge GraphsSimilar Papers 제목 키워드 기반
Calibration in Deep Learning: A Survey of the State-of-the-Art
Calibrating deep neural models plays an important role in building reliable, robust AI systems in safety-critical applications. Recent work has shown that modern neural networks that possess high predictive capability ar…
Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibration
In the last decade, machine learning techniques have gained popularity for estimating causal effects. One machine learning approach that can be used for estimating an average treatment effect is Double/debiased machine l…
Calibrating Bayesian UNet++ for Sub-Seasonal Forecasting
Seasonal forecasting is a crucial task when it comes to detecting the extreme heat and colds that occur due to climate change. Confidence in the predictions should be reliable since a small increase in the temperatures i…
Thermometer: Towards Universal Calibration for Large Language Models
We consider the issue of calibration in large language models (LLM). Recent studies have found that common interventions such as instruction tuning often result in poorly calibrated LLMs. Although calibration is well-exp…
Eigenvalue Calibration for Semantic Embeddings of Large Language Models
Uncertainty quantification is central to the reliable deployment of large language models (LLMs), and eigenvalues of semantic embeddings have recently emerged as a key tool in state-of-the-art methods. However, conventio…