Inter and Intra Topic Structure Learning with Word Embeddings
One important task of topic modeling for text analysis is interpretability. By discovering structured topics one is able to yield improved interpretability as well as modeling accuracy. In this paper, we propose a novel topic model with a deep structure that explores both inter-topic and intra-topic structures informed by word embeddings. Specifically, our model discovers inter topic structures in the form of topic hierarchies and discovers intra topic structures in the form of sub-topics, each of which is informed by word embeddings and captures a fine-grained thematic aspect of a normal topic. Extensive experiments demonstrate that our model achieves the state-of-the-art performance in terms of perplexity, document classification, and topic quality. Moreover, with topic hierarchies and sub-topics, the topics discovered in our model are more interpretable, providing an illuminating means to understand text data.
Code (1)
Tasks
Document ClassificationFormWord EmbeddingsMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Integrating Topic Modeling with Word Embeddings by Mixtures of vMFs
Gaussian LDA integrates topic modeling with word embeddings by replacing discrete topic distribution over word types with multivariate Gaussian distribution on the embedding space. This can take semantic information of w…
Topic ModelsWord EmbeddingsJointly Learning Word Embeddings and Latent Topics
Word embedding models such as Skip-gram learn a vector-space representation for each word, based on the local word collocation patterns that are observed in a text corpus. Latent topic models, on the other hand, take a m…
Learning Word EmbeddingsTopic ModelsWord EmbeddingsMulti-source Multi-view Transfer Learning in Neural Topic Modeling with Pretrained Topic and Word Embeddings
Though word embeddings and topics are complementary representations, several past works have only used pretrained word embeddings in (neural) topic modeling to address data sparsity problem in short text or small collect…
Information RetrievalRetrievalTransfer LearningWord EmbeddingsJOINTLY LEARNING TOPIC SPECIFIC WORD AND DOCUMENT EMBEDDING
Document embedding generally ignores underlying topics, which fails to capture polysemous terms that can mislead to improper thematic representation. Moreover, embedding a new document during the test process needs a com…
Document ClassificationDocument EmbeddingWord EmbeddingsA Neural Generative Model for Joint Learning Topics and Topic-Specific Word Embeddings
We propose a novel generative model to explore both local and global context for joint learning topics and topic-specific word embeddings. In particular, we assume that global latent topics are shared across documents, a…
Sentiment AnalysisSentiment ClassificationTopic ModelsWord Embeddings+2