UNSUPERVISED SENTENCE EMBEDDING USING DOCUMENT STRUCTURE-BASED CONTEXT
We present a new unsupervised method for learning general-purpose sentence embeddings. Unlike existing methods which rely on local contexts, such as words inside the sentence or immediately neighboring sentences, our method selects, for each target sentence, influential sentences in the entire document based on a document structure. We identify a dependency structure of sentences using metadata or text styles. Furthermore, we propose a novel out-of-vocabulary word handling technique to model many domain-specific terms, which were mostly discarded by existing sentence embedding methods. We validate our model on several tasks showing 30% precision improvement in coreference resolution in a technical domain, and 7.5% accuracy increase in paraphrase detection compared to baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
coreference-resolutionCoreference ResolutionSentenceSentence EmbeddingSentence-EmbeddingSentence EmbeddingsSimilar Papers 제목 키워드 기반
Unsupervised Contextualized Document Representation
Several NLP tasks need the effective representation of text documents. Arora et. al., 2017 demonstrate that simple weighted averaging of word vectors frequently outperforms neural models. SCDV (Mekala et. al., 2017) furt…
ClusteringSentenceSentence SimilarityWord Sense DisambiguationContextual Salience for Fast and Accurate Sentence Vectors
Unsupervised vector representations of sentences or documents are a major building block for many language tasks such as sentiment classification. However, current methods are uninterpretable and slow or require large tr…
Document SummarizationGeneral ClassificationSentenceSentence Embeddings+2Enhancing Unsupervised Sentence Similarity Methods with Deep Contextualised Word Representations
Calculating Semantic Textual Similarity (STS) plays a significant role in many applications such as question answering, document summarisation, information retrieval and information extraction. All modern state of the ar…
Contextualised Word RepresentationsInformation RetrievalQuestion AnsweringRetrieval+5Combining Word Embeddings and N-grams for Unsupervised Document Summarization
Graph-based extractive document summarization relies on the quality of the sentence similarity graph. Bag-of-words or tf-idf based sentence similarity uses exact word matching, but fails to measure the semantic similarit…
DiversityDocument SummarizationExtractive Document SummarizationExtractive Summarization+8GUSUM: Graph-Based Unsupervised Summarization using Sentence-BERT and Sentence Features
Unsupervised extractive document summarization aims to extract salient sentences from a document without requiring a labelled corpus. In existing graph-based methods, vertex and edge weights are mostly created by calcula…
Document SummarizationExtractive Document SummarizationExtractive Text SummarizationGraph Ranking+5