Deep Ensembles from a Bayesian Perspective
Deep ensembles can be considered as the current state-of-the-art for uncertainty quantification in deep learning. While the approach was originally proposed as a non-Bayesian technique, arguments supporting its Bayesian footing have been put forward as well. We show that deep ensembles can be viewed as an approximate Bayesian method by specifying the corresponding assumptions. Our findings lead to an improved approximation which results in an enlarged epistemic part of the uncertainty. Numerical examples suggest that the improved approximation can lead to more reliable uncertainties. Analytical derivations ensure easy calculation of results.
Code (0)
등록된 구현이 없습니다.
Tasks
Uncertainty QuantificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Deep Ensembles Secretly Perform Empirical Bayes
Quantifying uncertainty in neural networks is a highly relevant problem which is essential to many applications. The two predominant paradigms to tackle this task are Bayesian neural networks (BNNs) and deep ensembles. D…
A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods
We establish the first mathematically rigorous link between Bayesian, variational Bayesian, and ensemble methods. A key step towards this it to reformulate the non-convex optimisation problem typically encountered in dee…
Deep LearningUncertainty QuantificationVariational InferenceDeep Ensembles: A Loss Landscape Perspective
Deep ensembles have been empirically shown to be a promising approach for improving accuracy, uncertainty and out-of-distribution robustness of deep learning models. While deep ensembles were theoretically motivated by t…
Stein Variational Newton Neural Network Ensembles
Deep neural network ensembles are powerful tools for uncertainty quantification, which have recently been re-interpreted from a Bayesian perspective. However, current methods inadequately leverage second-order informatio…
Bayesian InferenceUncertainty QuantificationOn the Equivalence of Random Network Distillation, Deep Ensembles, and Bayesian Inference
Uncertainty quantification is central to safe and efficient deployments of deep learning models, yet many computationally practical methods lack lacking rigorous theoretical motivation. Random network distillation (RND) …
Bayesian Inference