Diversity-Aware Coherence Loss for Improving Neural Topic Models
The standard approach for neural topic modeling uses a variational autoencoder (VAE) framework that jointly minimizes the KL divergence between the estimated posterior and prior, in addition to the reconstruction loss. Since neural topic models are trained by recreating individual input documents, they do not explicitly capture the coherence between topic words on the corpus level. In this work, we propose a novel diversity-aware coherence loss that encourages the model to learn corpus-level coherence scores while maintaining a high diversity between topics. Experimental results on multiple datasets show that our method significantly improves the performance of neural topic models without requiring any pretraining or additional parameters.
Code (1)
Tasks
DiversityTopic ModelsSimilar Papers 제목 키워드 기반
Better Conversations by Modeling,Filtering,and Optimizing for Coherence and Diversity
We present three enhancements to existing encoder-decoder models for open-domain conversational agents, aimed at effectively modeling coherence and promoting output diversity: (1) We introduce a measure of coherence as t…
DecoderDiversityBetter Conversations by Modeling, Filtering, and Optimizing for Coherence and Diversity
We present three enhancements to existing encoder-decoder models for open-domain conversational agents, aimed at effectively modeling coherence and promoting output diversity: (1) We introduce a measure of coherence as t…
DecoderDialogue GenerationDiversityCoherence-Aware Neural Topic Modeling
Topic models are evaluated based on their ability to describe documents well (i.e. low perplexity) and to produce topics that carry coherent semantic meaning. In topic modeling so far, perplexity is a direct optimization…
Topic ModelsVariational InferenceHuman-Centric Topic Modeling with Goal-Prompted Contrastive Learning and Optimal Transport
Existing topic modeling methods, from LDA to recent neural and LLM-based approaches, which focus mainly on statistical coherence, often produce redundant or off-target topics that miss the user's underlying intent. We in…
Contrastive LearningWhen Numbers Tell Half the Story: Human-Metric Alignment in Topic Model Evaluation
Topic models uncover latent thematic structures in text corpora, yet evaluating their quality remains challenging, particularly in specialized domains. Existing methods often rely on automated metrics like topic coherenc…
Topic Models