Approximate Inference with the Variational Holder Bound
We introduce the Variational Holder (VH) bound as an alternative to Variational Bayes (VB) for approximate Bayesian inference. Unlike VB which typically involves maximization of a non-convex lower bound with respect to the variational parameters, the VH bound involves minimization of a convex upper bound to the intractable integral with respect to the variational parameters. Minimization of the VH bound is a convex optimization problem; hence the VH method can be applied using off-the-shelf convex optimization algorithms and the approximation error of the VH bound can also be analyzed using tools from convex optimization literature. We present experiments on the task of integrating a truncated multivariate Gaussian distribution and compare our method to VB, EP and a state-of-the-art numerical integration method for this problem.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceNumerical IntegrationSimilar Papers 제목 키워드 기반
Variational Inference with Holder Bounds
The recent introduction of thermodynamic integration techniques has provided a new framework for understanding and improving variational inference (VI). In this work, we present a careful analysis of the thermodynamic va…
Variational InferenceAmortized Inference of Variational Bounds for Learning Noisy-OR
Classical approaches for approximate inference depend on cleverly designed variational distributions and bounds. Modern approaches employ amortized variational inference, which uses a neural network to approximate any po…
Variational InferenceNested Variational Compression in Deep Gaussian Processes
Deep Gaussian processes provide a flexible approach to probabilistic modelling of data using either supervised or unsupervised learning. For tractable inference approximations to the marginal likelihood of the model must…
Gaussian ProcessesVariational InferenceRobust Variational Inference
Variational inference is a powerful tool for approximate inference. However, it mainly focuses on the evidence lower bound as variational objective and the development of other measures for variational inference is a pro…
Variational InferenceApproximate Variational Inference Based on a Finite Sample of Gaussian Latent Variables
Variational methods are employed in situations where exact Bayesian inference becomes intractable due to the difficulty in performing certain integrals. Typically, variational methods postulate a tractable posterior and …
Bayesian InferenceVariational Inference