Improving Confidence Estimates for Unfamiliar Examples
Intuitively, unfamiliarity should lead to lack of confidence. In reality, current algorithms often make highly confident yet wrong predictions when faced with relevant but unfamiliar examples. A classifier we trained to recognize gender is 12 times more likely to be wrong with a 99% confident prediction if presented with a subject from a different age group than those seen during training. In this paper, we compare and evaluate several methods to improve confidence estimates for unfamiliar and familiar samples. We propose a testing methodology of splitting unfamiliar and familiar samples by attribute (age, breed, subcategory) or sampling (similar datasets collected by different people at different times). We evaluate methods including confidence calibration, ensembles, distillation, and a Bayesian model and use several metrics to analyze label, likelihood, and calibration error. While all methods reduce over-confident errors, the ensemble of calibrated models performs best overall, and T-scaling performs best among the approaches with fastest inference. Our code is available at https://github.com/lizhitwo/ConfidenceEstimates . $\color{red}{\text{Please see UPDATED ERRATA.}}$
Code (1)
Tasks
AttributeDomain AdaptationSimilar Papers 제목 키워드 기반
Unfamiliar Finetuning Examples Control How Language Models Hallucinate
Large language models are known to hallucinate when faced with unfamiliar queries, but the underlying mechanism that govern how models hallucinate are not yet fully understood. In this work, we find that unfamiliar examp…
MMLUMultiple-choiceTriviaQAConversational Artificial Intelligence in Natural Language Processing Application with Lifelong Learning
Conversational AI bot known as chatbot has many roles in human life today, such as answering uncomplicated questions on E-commerce website pages or even help as an assistant like Siri and Google assistant. However, the c…
ChatbotLifelong learningYour Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator
Post-training of large language models is essential for adapting pre-trained language models (PLMs) to align with human preferences and downstream tasks. While PLMs typically exhibit well-calibrated confidence, post-trai…
Learning Confidence for Out-of-Distribution Detection in Neural Networks
Modern neural networks are very powerful predictive models, but they are often incapable of recognizing when their predictions may be wrong. Closely related to this is the task of out-of-distribution detection, where a n…
Out-of-Distribution DetectionFew-Shot Recalibration of Language Models
Recent work has uncovered promising ways to extract well-calibrated confidence estimates from language models (LMs), where the model's confidence score reflects how likely it is to be correct. However, while LMs may appe…
MathMMLU