Bin-wise Temperature Scaling (BTS): Improvement in Confidence Calibration Performance through Simple Scaling Techniques
The prediction reliability of neural networks is important in many applications. Specifically, in safety-critical domains, such as cancer prediction or autonomous driving, a reliable confidence of model's prediction is critical for the interpretation of the results. Modern deep neural networks have achieved a significant improvement in performance for many different image classification tasks. However, these networks tend to be poorly calibrated in terms of output confidence. Temperature scaling is an efficient post-processing-based calibration scheme and obtains well calibrated results. In this study, we leverage the concept of temperature scaling to build a sophisticated bin-wise scaling. Furthermore, we adopt augmentation of validation samples for elaborated scaling. The proposed methods consistently improve calibration performance with various datasets and deep convolutional neural network models.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous Drivingimage-classificationImage ClassificationPredictionSimilar Papers 제목 키워드 기반
Quantile Adaptive Temperature Scaling for Confidence Calibration
Deep neural networks often produce poorly calibrated confidence estimates, overstating their certainty even when predictions are incorrect. Temperature Scaling remains the most widely used posthoc calibration method due …
Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration
Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence. With neural networks, calibration can be improved by temperature scaling, a method to learn a single c…
Binary ClassificationBeyond temperature scaling: Obtaining well-calibrated multi-class probabilities with Dirichlet calibration
Class probabilities predicted by most multiclass classifiers are uncalibrated, often tending towards over-confidence. With neural networks, calibration can be improved by temperature scaling, a method to learn a single c…
Binary ClassificationDoes confidence calibration improve conformal prediction?
Conformal prediction is an emerging technique for uncertainty quantification that constructs prediction sets guaranteed to contain the true label with a predefined probability. Previous works often employ temperature sca…
Conformal PredictionPredictionUncertainty QuantificationSample-dependent Adaptive Temperature Scaling for Improved Calibration
It is now well known that neural networks can be wrong with high confidence in their predictions, leading to poor calibration. The most common post-hoc approach to compensate for this is to perform temperature scaling, w…
Out of Distribution (OOD) Detection