Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structure
Bayesian neural networks (BNNs) augment deep networks with uncertainty quantification by Bayesian treatment of the network weights. However, such models face the challenge of Bayesian inference in a high-dimensional and usually over-parameterized space. This paper investigates a new line of Bayesian deep learning by performing Bayesian inference on network structure. Instead of building structure from scratch inefficiently, we draw inspirations from neural architecture search to represent the network structure. We then develop an efficient stochastic variational inference approach which unifies the learning of both network structure and weights. Empirically, our method exhibits competitive predictive performance while preserving the benefits of Bayesian principles across challenging scenarios. We also provide convincing experimental justification for our modeling choice.
Code (1)
Tasks
Bayesian InferenceNeural Architecture SearchUncertainty QuantificationVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Uncertainty Quantification With Noise Injection in Neural Networks: A Bayesian Perspective
Model uncertainty quantification involves measuring and evaluating the uncertainty linked to a model's predictions, helping assess their reliability and confidence. Noise injection is a technique used to enhance the robu…
Bayesian InferenceUncertainty QuantificationBayesian Uncertainty Estimation for Batch Normalized Deep Networks
We show that training a deep network using batch normalization is equivalent to approximate inference in Bayesian models. We further demonstrate that this finding allows us to make meaningful estimates of the model uncer…
Qualitative Analysis of Monte Carlo Dropout
In this report, we present qualitative analysis of Monte Carlo (MC) dropout method for measuring model uncertainty in neural network (NN) models. We first consider the sources of uncertainty in NNs, and briefly review Ba…
Double-Bayesian Learning
Contemporary machine learning methods will try to approach the Bayes error, as it is the lowest possible error any model can achieve. This paper postulates that any decision is composed of not one but two Bayesian decisi…
Decision MakingMeasuring Classification Decision Certainty and Doubt
Quantitative characterizations and estimations of uncertainty are of fundamental importance in optimization and decision-making processes. Herein, we propose intuitive scores, which we call certainty and doubt, that can …
ClassificationDecision Making