Analysis of Variational Bayesian Latent Dirichlet Allocation: Weaker Sparsity Than MAP

2014-12-01 · NeurIPS 2014 12 · Shinichi Nakajima, Issei Sato, Masashi Sugiyama, Kazuho Watanabe, Hiroko Kobayashi

Latent Dirichlet allocation (LDA) is a popular generative model of various objects such as texts and images, where an object is expressed as a mixture of latent topics. In this paper, we theoretically investigate variational Bayesian (VB) learning in LDA. More specifically, we analytically derive the leading term of the VB free energy under an asymptotic setup, and show that there exist transition thresholds in Dirichlet hyperparameters around which the sparsity-inducing behavior drastically changes. Then we further theoretically reveal the notable phenomenon that VB tends to induce weaker sparsity than MAP in the LDA model, which is opposed to other models. We experimentally demonstrate the practical validity of our asymptotic theory on real-world Last.FM music data.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

LDA Linear discriminant analysis (LDA), normal discriminant analysis (NDA), or discriminant function analysis is a generalization of Fisher's linear discriminant, a method used in…

Stochastic Collapsed Variational Bayesian Inference for Latent Dirichlet Allocation

2013-05-10 · James Foulds, Levi Boyles, Christopher DuBois, Padhraic Smyth 외

In the internet era there has been an explosion in the amount of digital text information available, leading to difficulties of scale for traditional inference algorithms for topic models. Recent advances in stochastic v…

Bayesian Inference Topic Models Variational Inference

Algorithms of the LDA model [REPORT]

2013-07-01 · Jaka Špeh, Andrej Muhič, Jan Rupnik

We review three algorithms for Latent Dirichlet Allocation (LDA). Two of them are variational inference algorithms: Variational Bayesian inference and Online Variational Bayesian inference and one is Markov Chain Monte C…

Bayesian Inference model Variational Inference

Vectorized Bayesian Inference for Latent Dirichlet-Tree Allocation

2026-02-21 · Zheng Wang, Nizar Bouguila arxiv

Latent Dirichlet Allocation (LDA) is a foundational model for discovering latent thematic structure in discrete data, but its Dirichlet prior cannot represent the rich correlations and hierarchical relationships often pr…

Computational Efficiency Bayesian Inference

The Population Posterior and Bayesian Inference on Streams

2015-07-19 · James McInerney, Rajesh Ranganath, David M. Blei

Many modern data analysis problems involve inferences from streaming data. However, streaming data is not easily amenable to the standard probabilistic modeling approaches, which assume that we condition on finite data. …

Bayesian Inference

The Population Posterior and Bayesian Modeling on Streams

2015-12-01 · NeurIPS 2015 12 · James Mcinerney, Rajesh Ranganath, David Blei