Sampling-free Variational Inference for Neural Networks with Multiplicative Activation Noise
To adopt neural networks in safety critical domains, knowing whether we can trust their predictions is crucial. Bayesian neural networks (BNNs) provide uncertainty estimates by averaging predictions with respect to the posterior weight distribution. Variational inference methods for BNNs approximate the intractable weight posterior with a tractable distribution, yet mostly rely on sampling from the variational distribution during training and inference. Recent sampling-free approaches offer an alternative, but incur a significant parameter overhead. We here propose a more efficient parameterization of the posterior approximation for sampling-free variational inference that relies on the distribution induced by multiplicative Gaussian activation noise. This allows us to combine parameter efficiency with the benefits of sampling-free variational inference. Our approach yields competitive results for standard regression problems and scales well to large-scale image classification tasks including ImageNet.
Code (0)
등록된 구현이 없습니다.
Tasks
image-classificationImage ClassificationVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sampling-Free Variational Inference of Bayesian Neural Networks by Variance Backpropagation
We propose a new Bayesian Neural Net formulation that affords variational inference for which the evidence lower bound is analytically tractable subject to a tight approximation. We achieve this tractability by (i) decom…
Variational InferenceRobustly representing uncertainty in deep neural networks through sampling
As deep neural networks (DNNs) are applied to increasingly challenging problems, they will need to be able to represent their own uncertainty. Modeling uncertainty is one of the key features of Bayesian methods. Using Be…
Variational InferenceTruncation-free Hybrid Inference for DPMM
Dirichlet process mixture models (DPMM) are a cornerstone of Bayesian non-parametrics. While these models free from choosing the number of components a-priori, computationally attractive variational inference often reint…
Variational InferenceVariational Inference for Additive Main and Multiplicative Interaction Effects Models
In plant breeding the presence of a genotype by environment (GxE) interaction has a strong impact on cultivation decision making and the introduction of new crop cultivars. The combination of linear and bilinear terms ha…
Decision MakingVariational InferenceVariational Inference via Smoothed Particle Hydrodynamics
A new variational inference method, SPH-ParVI, based on smoothed particle hydrodynamics (SPH), is proposed for sampling partially known densities (e.g. up to a constant) or sampling using gradients. SPH-ParVI simulates t…
Bayesian InferenceVariational Inference