Evaluating Topic Quality with Posterior Variability
Probabilistic topic models such as latent Dirichlet allocation (LDA) are popularly used with Bayesian inference methods such as Gibbs sampling to learn posterior distributions over topic model parameters. We derive a novel measure of LDA topic quality using the variability of the posterior distributions. Compared to several existing baselines for automatic topic evaluation, the proposed metric achieves state-of-the-art correlations with human judgments of topic quality in experiments on three corpora. We additionally demonstrate that topic quality estimation can be further improved using a supervised estimator that combines multiple metrics.
Code (1)
Tasks
Bayesian InferenceTopic ModelsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Prior matters: simple and general methods for evaluating and improving topic quality in topic modeling
Latent Dirichlet Allocation (LDA) models trained without stopword removal often produce topics with high posterior probabilities on uninformative words, obscuring the underlying corpus content. Even when canonical stopwo…
Guaranteed inference in topic models
One of the core problems in statistical models is the estimation of a posterior distribution. For topic models, the problem of posterior inference for individual texts is particularly important, especially when dealing w…
Topic ModelsMachine Reading Tea Leaves: Automatically Evaluating Topic Coherence and Topic Model Quality
Evaluating Dynamic Topic Models
There is a lack of quantitative measures to evaluate the progression of topics through time in dynamic topic models (DTMs). Filling this gap, we propose a novel evaluation measure for DTMs that analyzes the changes in th…
Topic ModelsFast Learning of Clusters and Topics via Sparse Posteriors
Mixture models and topic models generate each observation from a single cluster, but standard variational posteriors for each observation assign positive probability to all possible clusters. This requires dense storage …
ArticlesTopic Models