Integrated Non-Factorized Variational Inference

2013-12-01 · NeurIPS 2013 12 · Shaobo Han, Xuejun Liao, Lawrence Carin

We present a non-factorized variational method for full posterior inference in Bayesian hierarchical models, with the goal of capturing the posterior variable dependencies via efficient and possibly parallel computation. Our approach unifies the integrated nested Laplace approximation (INLA) under the variational framework. The proposed method is applicable in more challenging scenarios than typically assumed by INLA, such as Bayesian Lasso, which is characterized by the non-differentiability of the $\ell_{1}$ norm arising from independent Laplace priors. We derive an upper bound for the Kullback-Leibler divergence, which yields a fast closed-form solution via decoupled optimization. Our method is a reliable analytic alternative to Markov chain Monte Carlo (MCMC), and it results in a tighter evidence lower bound than that of mean-field variational Bayes (VB) method.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Variational Inference

Variational Bayesian Inference with Stochastic Search

2012-06-27 · John Paisley, David Blei, Michael Jordan

Mean-field variational inference is a method for approximate Bayesian posterior inference. It approximates a full posterior distribution with a factorized set of distributions by maximizing a lower bound on the marginal …

Bayesian Inference Stochastic Optimization Variational Inference

Noisy Natural Gradient as Variational Inference

2017-12-06 · ICML 2018 7 · Guodong Zhang, Shengyang Sun, David Duvenaud, Roger Grosse

Variational Bayesian neural nets combine the flexibility of deep learning with Bayesian uncertainty estimation. Unfortunately, there is a tradeoff between cheap but simple variational families (e.g.~fully factorized) or …

Active Learning Efficient Exploration Reinforcement Learning Reinforcement Learning (RL)+1

Efficient Approximate Inference with Walsh-Hadamard Variational Inference

2019-11-29 · Simone Rossi, Sebastien Marmin, Maurizio Filippone

Variational inference offers scalable and flexible tools to tackle intractable Bayesian inference of modern statistical models like Bayesian neural networks and Gaussian processes. For largely over-parameterized models, …

Bayesian Inference Gaussian Processes Variational Inference

Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces

2019-05-17 · Philipp Becker, Harit Pandya, Gregor Gebhardt, Cheng Zhao 외

In order to integrate uncertainty estimates into deep time-series modelling, Kalman Filters (KFs) (Kalman et al., 1960) have been integrated with deep learning models, however, such approaches typically rely on approxima…

Image Imputation Imputation Time Series Time Series Analysis+2

Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs

2024-03-20 · Charles C. Margossian, Loucas Pillaud-Vivien, Lawrence K. Saul

Given an intractable distribution $p$, the problem of variational inference (VI) is to find the best approximation from some more tractable family $Q$. Commonly, one chooses $Q$ to be a family of factorized distributions…

Uncertainty Quantification valid Variational Inference