Modeling Text using the Continuous Space Topic Model with Pre-Trained Word Embeddings
In this study, we propose a model that extends the continuous space topic model (CSTM), which flexibly controls word probability in a document, using pre-trained word embeddings. To develop the proposed model, we pre-train word embeddings, which capture the semantics of words and plug them into the CSTM. Intrinsic experimental results show that the proposed model exhibits a superior performance over the CSTM in terms of perplexity and convergence speed. Furthermore, extrinsic experimental results show that the proposed model is useful for a document classification task when compared with the baseline model. We qualitatively show that the latent coordinates obtained by training the proposed model are better than those of the baseline model.
Code (0)
등록된 구현이 없습니다.
Tasks
Document ClassificationWord EmbeddingsSimilar Papers 제목 키워드 기반
Generative Topic Embedding: a Continuous Representation of Documents (Extended Version with Proofs)
Word embedding maps words into a low-dimensional continuous embedding space by exploiting the local word collocation patterns in a small context window. On the other hand, topic modeling maps documents onto a low-dimensi…
Document ClassificationVariational InferenceMulti-source Neural Topic Modeling in Multi-view Embedding Spaces
Though word embeddings and topics are complementary representations, several past works have only used pretrained word embeddings in (neural) topic modeling to address data sparsity in short-text or small collection of d…
Information RetrievalRetrievalWord EmbeddingsCobwebTM: Probabilistic Concept Formation for Lifelong and Hierarchical Topic Modeling
Topic modeling seeks to uncover latent semantic structure in text corpora with minimal supervision. Neural approaches achieve strong performance but require extensive tuning and struggle with lifelong learning due to cat…
Capturing research literature attitude towards Sustainable Development Goals: an LLM-based topic modeling approach
The world is facing a multitude of challenges that hinder the development of human civilization and the well-being of humanity on the planet. The Sustainable Development Goals (SDGs) were formulated by the United Nations…
GloCTM: Cross-Lingual Topic Modeling via a Global Context Space
Cross-lingual topic modeling seeks to uncover coherent and semantically aligned topics across languages - a task central to multilingual understanding. Yet most existing models learn topics in disjoint, language-specific…