Bayesian Paragraph Vectors
Word2vec (Mikolov et al., 2013) has proven to be successful in natural language processing by capturing the semantic relationships between different words. Built on top of single-word embeddings, paragraph vectors (Le and Mikolov, 2014) find fixed-length representations for pieces of text with arbitrary lengths, such as documents, paragraphs, and sentences. In this work, we propose a novel interpretation for neural-network-based paragraph vectors by developing an unsupervised generative model whose maximum likelihood solution corresponds to traditional paragraph vectors. This probabilistic formulation allows us to go beyond point estimates of parameters and to perform Bayesian posterior inference. We find that the entropy of paragraph vectors decreases with the length of documents, and that information about posterior uncertainty improves performance in supervised learning tasks such as sentiment analysis and paraphrase detection.
Code (0)
등록된 구현이 없습니다.
Tasks
Sentiment AnalysisWord EmbeddingsSimilar Papers 제목 키워드 기반
Specialising Paragraph Vectors for Text Polarity Detection
This paper presents some experiments for specialising Paragraph Vectors, a new technique for creating text fragment (phrase, sentence, paragraph, text, ...) embedding vectors, for text polarity detection. The first exten…
regressionSentenceMulti-Document Summarization of Persian Text using Paragraph Vectors
A multi-document summarizer finds the key topics from multiple textual sources and organizes information around them. In this paper we propose a summarization method for Persian text using paragraph vectors that can repr…
Abstractive Text SummarizationDocument SummarizationMulti-Document SummarizationDocument Embedding with Paragraph Vectors
Paragraph Vectors has been recently proposed as an unsupervised method for learning distributed representations for pieces of texts. In their work, the authors showed that the method can learn an embedding of movie revie…
Document EmbeddingSentiment AnalysisWord EmbeddingsBinary Paragraph Vectors
Recently Le & Mikolov described two log-linear models, called Paragraph Vector, that can be used to learn state-of-the-art distributed representations of documents. Inspired by this work, we present Binary Paragraph Vect…
Information RetrievalRetrievalTransfer LearningDiverse and Coherent Paragraph Generation from Images
Paragraph generation from images, which has gained popularity recently, is an important task for video summarization, editing, and support of the disabled. Traditional image captioning methods fall short on this front, s…
DiversityImage CaptioningImage Paragraph CaptioningVideo Summarization