Neural Sparse Topical Coding
Topic models with sparsity enhancement have been proven to be effective at learning discriminative and coherent latent topics of short texts, which is critical to many scientific and engineering applications. However, the extensions of these models require carefully tailored graphical models and re-deduced inference algorithms, limiting their variations and applications. We propose a novel sparsity-enhanced topic model, Neural Sparse Topical Coding (NSTC) base on a sparsity-enhanced topic model called Sparse Topical Coding (STC). It focuses on replacing the complex inference process with the back propagation, which makes the model easy to explore extensions. Moreover, the external semantic information of words in word embeddings is incorporated to improve the representation of short texts. To illustrate the flexibility offered by the neural network based framework, we present three extensions base on NSTC without re-deduced inference algorithms. Experiments on Web Snippet and 20Newsgroups datasets demonstrate that our models outperform existing methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingTopic ModelsWord EmbeddingsSimilar Papers 제목 키워드 기반
Topical Language Generation using Transformers
Large-scale transformer-based language models (LMs) demonstrate impressive capabilities in open text generation. However, controlling the generated text's properties such as the topic, style, and sentiment is challenging…
DiversityText GenerationExploring the Role of Diversity in Example Selection for In-Context Learning
In-Context Learning (ICL) has gained prominence due to its ability to perform tasks without requiring extensive training data and its robustness to noisy labels. A typical ICL workflow involves selecting localized exampl…
DiversityIn-Context LearningRerankingTopic-relevant Response Generation using Optimal Transport for an Open-domain Dialog System
Conventional neural generative models tend to generate safe and generic responses which have little connection with previous utterances semantically and would disengage users in a dialog system. To generate relevant resp…
Open-Domain DialogResponse GenerationModeling Topical Relevance for Multi-Turn Dialogue Generation
Topic drift is a common phenomenon in multi-turn dialogue. Therefore, an ideal dialogue generation models should be able to capture the topic information of each context, detect the relevant context, and produce appropri…
Dialogue GenerationSentenceTopical: Learning Repository Embeddings from Source Code using Attention
This paper presents Topical, a novel deep neural network for repository level embeddings. Existing methods, reliant on natural language documentation or naive aggregation techniques, are outperformed by Topical's utiliza…