Variational Inference for Latent Variable Models in High Dimensions
Variational inference (VI) is a popular method for approximating intractable posterior distributions in Bayesian inference and probabilistic machine learning. In this paper, we introduce a general framework for quantifying the statistical accuracy of mean-field variational inference (MFVI) for posterior approximation in Bayesian latent variable models with categorical local latent variables. Utilizing our general framework, we capture the exact asymptotic regime where MFVI `works' for the celebrated latent Dirichlet allocation (LDA) model. Focusing on the mixed membership stochastic blockmodel (MMSB), we show that the vanilla fully factorized MFVI, often used in the literature, is suboptimal. We propose a partially grouped VI algorithm for this model and show that it works, and derive its exact asymptotic performance. We further illustrate that our bounds are tight for both the above models.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Latent Variables on Spheres for Sampling and Inference
Variational inference is a fundamental problem in Variational AutoEncoder (VAE). The optimization with lower bound of marginal log-likelihood results in the distribution of latent variables approximate to a given prior p…
Variational InferenceLatent Variables on Spheres for Autoencoders in High Dimensions
Variational Auto-Encoder (VAE) has been widely applied as a fundamental generative model in machine learning. For complex samples like imagery objects or scenes, however, VAE suffers from the dimensional dilemma between …
Vocal Bursts Intensity PredictionHierarchical Variational Models
Black box variational inference allows researchers to easily prototype and evaluate an array of models. Recent advances allow such algorithms to scale to high dimensions. However, a central question remains: How to speci…
Computational EfficiencyVariational InferenceStructured Kernel Regression VAE: A Computationally Efficient Surrogate for GP-VAEs in ICA
The interpretability of generative models is considered a key factor in demonstrating their effectiveness and controllability. The generated data are believed to be determined by latent variables that are not directly ob…
Computational EfficiencyBayesian InferenceGaussian ProcessesCausal InferencetvGP-VAE: Tensor-variate Gaussian Process Prior Variational Autoencoder
Variational autoencoders (VAEs) are a powerful class of deep generative latent variable model for unsupervised representation learning on high-dimensional data. To ensure computational tractability, VAEs are often implem…
Gaussian ProcessesRepresentation LearningTime SeriesTime Series Analysis