Improving robustness and calibration in ensembles with diversity regularization
Calibration and uncertainty estimation are crucial topics in high-risk environments. We introduce a new diversity regularizer for classification tasks that uses out-of-distribution samples and increases the overall accuracy, calibration and out-of-distribution detection capabilities of ensembles. Following the recent interest in the diversity of ensembles, we systematically evaluate the viability of explicitly regularizing ensemble diversity to improve calibration on in-distribution data as well as under dataset shift. We demonstrate that diversity regularization is highly beneficial in architectures, where weights are partially shared between the individual members and even allows to use fewer ensemble members to reach the same level of robustness. Experiments on CIFAR-10, CIFAR-100, and SVHN show that regularizing diversity can have a significant impact on calibration and robustness, as well as out-of-distribution detection.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityOut-of-Distribution DetectionSimilar Papers 제목 키워드 기반
Diversity regularization in deep ensembles
Calibrating the confidence of supervised learning models is important for a variety of contexts where the certainty over predictions should be reliable. However, it has been reported that deep neural network models are o…
DiversityPredictionNeural Ensemble Search for Uncertainty Estimation and Dataset Shift
Ensembles of neural networks achieve superior performance compared to stand-alone networks in terms of accuracy, uncertainty calibration and robustness to dataset shift. \emph{Deep ensembles}, a state-of-the-art method f…
AutoMLDiversityImage ClassificationNeural Architecture SearchPacked-Ensembles for Efficient Uncertainty Estimation
Deep Ensembles (DE) are a prominent approach for achieving excellent performance on key metrics such as accuracy, calibration, uncertainty estimation, and out-of-distribution detection. However, hardware limitations of r…
Classifier calibrationDiversityImage ClassificationOut-of-Distribution Detection+1Input-gradient space particle inference for neural network ensembles
Deep Ensembles (DEs) demonstrate improved accuracy, calibration and robustness to perturbations over single neural networks partly due to their functional diversity. Particle-based variational inference (ParVI) methods e…
DiversityEnsemble Learningimage-classificationImage Classification+2Hyperparameter Ensembles for Robustness and Uncertainty Quantification
Ensembles over neural network weights trained from different random initialization, known as deep ensembles, achieve state-of-the-art accuracy and calibration. The recently introduced batch ensembles provide a drop-in re…
Diversityimage-classificationImage ClassificationUncertainty Quantification