Communication-Free Parallel Supervised Topic Models
Embarrassingly (communication-free) parallel Markov chain Monte Carlo (MCMC) methods are commonly used in learning graphical models. However, MCMC cannot be directly applied in learning topic models because of the quasi-ergodicity problem caused by multimodal distribution of topics. In this paper, we develop an embarrassingly parallel MCMC algorithm for sLDA. Our algorithm works by switching the order of sampled topics combination and labeling variable prediction in sLDA, in which it overcomes the quasi-ergodicity problem because high-dimension topics that follow a multimodal distribution are projected into one-dimension document labels that follow a unimodal distribution. Our empirical experiments confirm that the out-of-sample prediction performance using our embarrassingly parallel algorithm is comparable to non-parallel sLDA while the computation time is significantly reduced.
Code (0)
등록된 구현이 없습니다.
Tasks
Topic ModelsSimilar Papers 제목 키워드 기반
Towards Big Topic Modeling
To solve the big topic modeling problem, we need to reduce both time and space complexities of batch latent Dirichlet allocation (LDA) algorithms. Although parallel LDA algorithms on the multi-processor architecture have…
Communication-free Sampling and 4D Hybrid Parallelism for Scalable Mini-batch GNN Training
Graph neural networks (GNNs) are widely used for learning on graph datasets derived from various real-world scenarios. Learning from extremely large graphs requires distributed training, and mini-batching with sampling i…
Unsupervised Machine Commenting with Neural Variational Topic Model
Article comments can provide supplementary opinions and facts for readers, thereby increase the attraction and engagement of articles. Therefore, automatically commenting is helpful in improving the activeness of the com…
ArticlesmodelRetrievalScaling up Dynamic Topic Models
Dynamic topic models (DTMs) are very effective in discovering topics and capturing their evolution trends in time series data. To do posterior inference of DTMs, existing methods are all batch algorithms that scan the fu…
Time SeriesTime Series AnalysisTopic ModelsEffect Of Weather Conditions On FSO Link
Free Space Optics (FSO) is a developing technology for Line of Sight communication that uses light propagation in free space that provides various advantages like high bandwidth, high data rate, ease of installation, fre…
Sand