Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
We show that training a deep network using batch normalization is equivalent to approximate inference in Bayesian models. We further demonstrate that this finding allows us to make meaningful estimates of the model uncertainty using conventional architectures, without modifications to the network or the training procedure. Our approach is thoroughly validated by measuring the quality of uncertainty in a series of empirical experiments on different tasks. It outperforms baselines with strong statistical significance, and displays competitive performance with recent Bayesian approaches.
Code (4)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fast Bayesian Uncertainty Estimation and Reduction of Batch Normalized Single Image Super-Resolution Network
Convolutional neural network (CNN) has achieved unprecedented success in image super-resolution tasks in recent years. However, the network's performance depends on the distribution of the training sets and degrades on o…
Adversarial DefenseImage ReconstructionImage Super-ResolutionSuper-ResolutionAmortized Conditional Normalized Maximum Likelihood: Reliable Out of Distribution Uncertainty Estimation
While deep neural networks provide good performance for a range of challenging tasks, calibration and uncertainty estimation remain major challenges, especially under distribution shift. In this paper, we propose the amo…
Bayesian InferenceAmortized Conditional Normalized Maximum Likelihood
While deep neural networks provide good performance for a range of challenging tasks, calibration and uncertainty estimation remain major challenges. In this paper, we propose the amortized conditional normalized maximum…
Bayesian InferenceStochastic Normalizations as Bayesian Learning
In this work we investigate the reasons why Batch Normalization (BN) improves the generalization performance of deep networks. We argue that one major reason, distinguishing it from data-independent normalization methods…
Batch Bayesian optimisation via density-ratio estimation with guarantees
Bayesian optimisation (BO) algorithms have shown remarkable success in applications involving expensive black-box functions. Traditionally BO has been set as a sequential decision-making process which estimates the utili…
Bayesian InferenceBayesian OptimisationDecision MakingDensity Ratio Estimation+1