Improving a tf-idf weighted document vector embedding
We examine a number of methods to compute a dense vector embedding for a document in a corpus, given a set of word vectors such as those from word2vec or GloVe. We describe two methods that can improve upon a simple weighted sum, that are optimal in the sense that they maximizes a particular weighted cosine similarity measure. We consider several weighting functions, including inverse document frequency (idf), smooth inverse frequency (SIF), and the sub-sampling function used in word2vec. We find that idf works best for our applications. We also use common component removal proposed by Arora et al. as a post-process and find it is helpful in most cases. We compare these embeddings variations to the doc2vec embedding on a new evaluation task using TripAdvisor reviews, and also on the CQADupStack benchmark from the literature.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sentiment Classification Using Document Embeddings Trained with Cosine Similarity
In document-level sentiment classification, each document must be mapped to a fixed length vector. Document embedding models map each document to a dense, low-dimensional vector in continuous vector space. This paper pro…
ClassificationDocument EmbeddingGeneral ClassificationSentiment Analysis+1JOINTLY LEARNING TOPIC SPECIFIC WORD AND DOCUMENT EMBEDDING
Document embedding generally ignores underlying topics, which fails to capture polysemous terms that can mislead to improper thematic representation. Moreover, embedding a new document during the test process needs a com…
Document ClassificationDocument EmbeddingWord EmbeddingsP-SIF: Document Embeddings Using Partition Averaging
Simple weighted averaging of word vectors often yields effective representations for sentences which outperform sophisticated seq2seq neural models in many tasks. While it is desirable to use the same method to represent…
Improving Document Classification with Multi-Sense Embeddings
Efficient representation of text documents is an important building block in many NLP tasks. Research on long text categorization has shown that simple weighted averaging of word vectors for sentence representation often…
ClassificationClusteringDocument ClassificationGeneral Classification+3Unsupervised Document Representation using Partition Word-Vectors Averaging
Learning effective document-level representation is essential in many important NLP tasks such as document classification, summarization, etc. Recent research has shown that simple weighted averaging of word vectors is a…
Document ClassificationSentence