Structured Black Box Variational Inference for Latent Time Series Models
Continuous latent time series models are prevalent in Bayesian modeling; examples include the Kalman filter, dynamic collaborative filtering, or dynamic topic models. These models often benefit from structured, non mean field variational approximations that capture correlations between time steps. Black box variational inference with reparameterization gradients (BBVI) allows us to explore a rich new class of Bayesian non-conjugate latent time series models; however, a naive application of BBVI to such structured variational models would scale quadratically in the number of time steps. We describe a BBVI algorithm analogous to the forward-backward algorithm which instead scales linearly in time. It allows us to efficiently sample from the variational distribution and estimate the gradients of the ELBO. Finally, we show results on the recently proposed dynamic word embedding model, which was trained using our method.
Code (0)
등록된 구현이 없습니다.
Tasks
Collaborative FilteringTime SeriesTime Series AnalysisTopic ModelsVariational InferenceSimilar Papers 제목 키워드 기반
Black box variational inference for state space models
Latent variable time-series models are among the most heavily used tools from machine learning and applied statistics. These models have the advantage of learning latent structure both from noisy observations and from th…
State Space ModelsTime SeriesTime Series AnalysisVariational InferenceDenoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors
We propose denoising diffusion variational inference (DDVI), a black-box variational inference algorithm for latent variable models which relies on diffusion models as flexible approximate posteriors. Specifically, our m…
DenoisingVariational InferenceBlack Box Variational Inference
Variational inference has become a widely used method to approximate posteriors in complex latent variables models. However, deriving a variational inference algorithm generally requires significant model-specific analys…
Stochastic OptimizationVariational InferenceStructured Optimal Variational Inference for Dynamic Latent Space Models
We consider a latent space model for dynamic networks, where our objective is to estimate the pairwise inner products plus the intercept of the latent positions. To balance posterior inference and computational scalabili…
Variational InferenceA derivation of variational message passing (VMP) for latent Dirichlet allocation (LDA)
Latent Dirichlet Allocation (LDA) is a probabilistic model used to uncover latent topics in a corpus of documents. Inference is often performed using variational Bayes (VB) algorithms, which calculate a lower bound to th…
Variational Inference