Variational Inference on the Final-Layer Output of Neural Networks
Traditional neural networks are simple to train but they typically produce overconfident predictions. In contrast, Bayesian neural networks provide good uncertainty quantification but optimizing them is time consuming due to the large parameter space. This paper proposes to combine the advantages of both approaches by performing Variational Inference in the Final layer Output space (VIFO), because the output space is much smaller than the parameter space. We use neural networks to learn the mean and the variance of the probabilistic output. Using the Bayesian formulation we incorporate collapsed variational inference with VIFO which significantly improves the performance in practice. On the other hand, like standard, non-Bayesian models, VIFO enjoys simple training and one can use Rademacher complexity to provide risk bounds for the model. Experiments show that VIFO provides a good tradeoff in terms of run time and uncertainty quantification, especially for out of distribution data.
Code (1)
Tasks
Uncertainty QuantificationVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Uncertainty Estimations by Softplus normalization in Bayesian Convolutional Neural Networks with Variational Inference
We introduce a novel uncertainty estimation for classification tasks for Bayesian convolutional neural networks with variational inference. By normalizing the output of a Softplus function in the final layer, we estimate…
Bayesian InferenceGeneral ClassificationVariational InferenceDifferentiable Learning of Submodular Models
Can we incorporate discrete optimization algorithms within modern machine learning models? For example, is it possible to use in deep architectures a layer whose output is the minimal cut of a parametrized graph? Given t…
Variational InferenceVariational Bayesian Last Layers
We introduce a deterministic variational formulation for training Bayesian last layer neural networks. This yields a sampling-free, single-pass model and loss that effectively improves uncertainty estimation. Our variati…
Out-of-Distribution DetectionVariational InferenceLadder Variational Autoencoders
Variational Autoencoders are powerful models for unsupervised learning. However deep models with several layers of dependent stochastic variables are difficult to train which limits the improvements obtained using these …
Unsupervised MNISTLearning a Model-Driven Variational Network for Deformable Image Registration
Data-driven deep learning approaches to image registration can be less accurate than conventional iterative approaches, especially when training data is limited. To address this whilst retaining the fast inference speed …
Deep LearningDenoisingImage Registration