Calibrating Where It Matters: Constrained Temperature Scaling
We consider calibration of convolutional classifiers for diagnostic decision making. Clinical decision makers can use calibrated classifiers to minimise expected costs given their own cost function. Such functions are usually unknown at training time. If minimising expected costs is the primary aim, algorithms should focus on tuning calibration in regions of probability simplex likely to effect decisions. We give an example, modifying temperature scaling calibration, and demonstrate improved calibration where it matters using convnets trained to classify dermoscopy images.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingDiagnosticMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving the Post-hoc Calibration of Modern Neural Networks with Probe Scaling
We present "probe scaling": a post-hoc recipe for calibrating the predictions of modern neural networks. Our recipe is inspired by several lines of work, which demonstrate that early layers in the neural network learn ge…
Temperature Scaling for Quantile Calibration
Deep learning models are often poorly calibrated, i.e., they may produce overconfident predictions that are wrong, implying that their uncertainty estimates are unreliable. While a number of approaches have been propos…
ClassificationregressionLong Horizon Temperature Scaling
Temperature scaling is a popular technique for tuning the sharpness of a model distribution. It is used extensively for sampling likely generations and calibrating model uncertainty, and even features as a controllable p…
Multiple-choiceAn Empirical Study Into What Matters for Calibrating Vision-Language Models
Vision-Language Models (VLMs) have emerged as the dominant approach for zero-shot recognition, adept at handling diverse scenarios and significant distribution changes. However, their deployment in risk-sensitive areas r…
Zero-Shot LearningTemperature-Free Loss Function for Contrastive Learning
As one of the most promising methods in self-supervised learning, contrastive learning has achieved a series of breakthroughs across numerous fields. A predominant approach to implementing contrastive learning is applyin…
Contrastive LearningSelf-Supervised Learningvalid