Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks
Deep Neural Networks (DNNs) have achieved state-of-the-art accuracy performance in many tasks. However, recent works have pointed out that the outputs provided by these models are not well-calibrated, seriously limiting their use in critical decision scenarios. In this work, we propose to use a decoupled Bayesian stage, implemented with a Bayesian Neural Network (BNN), to map the uncalibrated probabilities provided by a DNN to calibrated ones, consistently improving calibration. Our results evidence that incorporating uncertainty provides more reliable probabilistic models, a critical condition for achieving good calibration. We report a generous collection of experimental results using high-accuracy DNNs in standardized image classification benchmarks, showing the good performance, flexibility and robust behavior of our approach with respect to several state-of-the-art calibration methods. Code for reproducibility is provided.
Code (1)
Tasks
image-classificationImage ClassificationSimilar Papers 제목 키워드 기반
Bayesian Calibration of MEMS Accelerometers
This study aims to investigate the utilization of Bayesian techniques for the calibration of micro-electro-mechanical systems (MEMS) accelerometers. These devices have garnered substantial interest in various practical a…
Probabilistic ProgrammingBayesian Neural Network Versus Ex-Post Calibration For Prediction Uncertainty
Probabilistic predictions from neural networks which account for predictive uncertainty during classification is crucial in many real-world and high-impact decision making settings. However, in practice most datasets are…
Decision MakingActive Uncertainty Calibration in Bayesian ODE Solvers
There is resurging interest, in statistics and machine learning, in solvers for ordinary differential equations (ODEs) that return probability measures instead of point estimates. Recently, Conrad et al. introduced a sam…
Epidemiological Model Calibration via Graybox Bayesian Optimization
In this study, we focus on developing efficient calibration methods via Bayesian decision-making for the family of compartmental epidemiological models. The existing calibration methods usually assume that the compartmen…
Bayesian OptimizationDecision MakingGaussian ProcessesmodelBayesian calibration for forensic evidence reporting
We introduce a Bayesian solution for the problem in forensic speaker recognition, where there may be very little background material for estimating score calibration parameters. We work within the Bayesian paradigm of ev…
Speaker Recognition