On Representations of Mean-Field Variational Inference
The mean field variational inference (MFVI) formulation restricts the general Bayesian inference problem to the subspace of product measures. We present a framework to analyze MFVI algorithms, which is inspired by a similar development for general variational Bayesian formulations. Our approach enables the MFVI problem to be represented in three different manners: a gradient flow on Wasserstein space, a system of Fokker-Planck-like equations and a diffusion process. Rigorous guarantees are established to show that a time-discretized implementation of the coordinate ascent variational inference algorithm in the product Wasserstein space of measures yields a gradient flow in the limit. A similar result is obtained for their associated densities, with the limit being given by a quasi-linear partial differential equation. A popular class of practical algorithms falls in this framework, which provides tools to establish convergence. We hope this framework could be used to guarantee convergence of algorithms in a variety of approaches, old and new, to solve variational inference problems.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Particle Algorithm for Mean-Field Variational Inference
Variational inference is a fast and scalable alternative to Markov chain Monte Carlo and has been widely applied to posterior inference tasks in statistics and machine learning. A traditional approach for implementing me…
Variational InferenceTheoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection
The mean field variational Bayes method is becoming increasingly popular in statistics and machine learning. Its iterative Coordinate Ascent Variational Inference algorithm has been widely applied to large scale Bayesian…
Bayesian InferenceCommunity DetectionStochastic Block ModelVariational InferenceWide Mean-Field Variational Bayesian Neural Networks Ignore the Data
Variational inference enables approximate posterior inference of the highly over-parameterized neural networks that are popular in modern machine learning. Unfortunately, such posteriors are known to exhibit various path…
BIG-bench Machine LearningVariational InferenceBeta Process Non-negative Matrix Factorization with Stochastic Structured Mean-Field Variational Inference
Beta process is the standard nonparametric Bayesian prior for latent factor model. In this paper, we derive a structured mean-field variational inference algorithm for a beta process non-negative matrix factorization (NM…
Variational InferenceStatistical Inference in Mean-Field Variational Bayes
We conduct non-asymptotic analysis on the mean-field variational inference for approximating posterior distributions in complex Bayesian models that may involve latent variables. We show that the mean-field approximation…
Variational Inference