Improving Document Classification with Multi-Sense Embeddings
Efficient representation of text documents is an important building block in many NLP tasks. Research on long text categorization has shown that simple weighted averaging of word vectors for sentence representation often outperforms more sophisticated neural models. Recently proposed Sparse Composite Document Vector (SCDV) (Mekala et. al, 2017) extends this approach from sentences to documents using soft clustering over word vectors. However, SCDV disregards the multi-sense nature of words, and it also suffers from the curse of higher dimensionality. In this work, we address these shortcomings and propose SCDV-MS. SCDV-MS utilizes multi-sense word embeddings and learns a lower dimensional manifold. Through extensive experiments on multiple real-world datasets, we show that SCDV-MS embeddings outperform previous state-of-the-art embeddings on multi-class and multi-label text categorization tasks. Furthermore, SCDV-MS embeddings are more efficient than SCDV in terms of time and space complexity on textual classification tasks.
Code (1)
Tasks
ClassificationClusteringDocument ClassificationGeneral ClassificationSentenceText CategorizationWord EmbeddingsSimilar Papers 제목 키워드 기반
Document-level Neural Machine Translation with Document Embeddings
Standard neural machine translation (NMT) is on the assumption of document-level context independent. Most existing document-level NMT methods are satisfied with a smattering sense of brief document-level information, wh…
Machine TranslationNMTTranslationDocument Classification with Word Sense Knowledge
The performance of Word Sense Disambiguation (WSD) on a standard evaluation framework has reached an estimated upper bound. However, there is limited research on the application of WSD to relevant NLP tasks due to the hi…
ClassificationDocument ClassificationWord Sense DisambiguationWord Sense Disambiguation: Towards Interactive Context Exploitation from Both Word and Sense Perspectives
Lately proposed Word Sense Disambiguation (WSD) systems have approached the estimated upper bound of the task on standard evaluation benchmarks. However, these systems typically implement the disambiguation of words in a…
SentenceWord Sense DisambiguationVector representations of text data in deep learning
In this dissertation we report results of our research on dense distributed representations of text data. We propose two novel neural models for learning such representations. The first model learns representations at th…
Deep LearningInformation RetrievalPart-Of-Speech TaggingRetrieval+3Specialized Document Embeddings for Aspect-based Similarity of Research Papers
Document embeddings and similarity measures underpin content-based recommender systems, whereby a document is commonly represented as a single generic embedding. However, similarity computed on single vector representati…
Document ClassificationRecommendation SystemsSegmentation