Scalable Inference for Logistic-Normal Topic Models
Logistic-normal topic models can effectively discover correlation structures among latent topics. However, their inference remains a challenge because of the non-conjugacy between the logistic-normal prior and multinomial topic mixing proportions. Existing algorithms either make restricting mean-field assumptions or are not scalable to large-scale applications. This paper presents a partially collapsed Gibbs sampling algorithm that approaches the provably correct distribution by exploring the ideas of data augmentation. To improve time efficiency, we further present a parallel implementation that can deal with large-scale applications and learn the correlation structures of thousands of topics from millions of documents. Extensive empirical results demonstrate the promise.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationTopic ModelsSimilar Papers 제목 키워드 기반
Probit Normal Correlated Topic Models
The logistic normal distribution has recently been adapted via the transformation of multivariate Gaus- sian variables to model the topical distribution of documents in the presence of correlations among topics. In this …
Topic ModelsScalable Inference for Bayesian Multinomial Logistic-Normal Dynamic Linear Models
Many scientific fields collect longitudinal count compositional data. Each observation is a multivariate count vector, where the total counts are arbitrary, and the information lies in the relative frequency of the count…
State EstimationNeural Topic Modeling with Bidirectional Adversarial Training
Recent years have witnessed a surge of interests of using neural topic models for automatic topic extraction from text, since they avoid the complicated mathematical derivations for model inference as in traditional topi…
ClusteringText ClusteringTopic ModelsDiffETM: Diffusion Process Enhanced Embedded Topic Model
The embedded topic model (ETM) is a widely used approach that assumes the sampled document-topic distribution conforms to the logistic normal distribution for easier optimization. However, this assumption oversimplifies …
modelImproved Bayesian Logistic Supervised Topic Models with Data Augmentation
Supervised topic models with a logistic likelihood have two issues that potentially limit their practical use: 1) response variables are usually over-weighted by document word counts; and 2) existing variational inferenc…
Bayesian InferenceData AugmentationTopic ModelsVariational Inference