Do sentence embeddings capture discourse properties of sentences from Scientific Abstracts ?
We introduce four tasks designed to determine which sentence encoders best capture discourse properties of sentences from scientific abstracts, namely coherence and cohesion between clauses of a sentence, and discourse relations within sentences. We show that even if contextual encoders such as BERT or SciBERT encodes the coherence in discourse units, they do not help to predict three discourse relations commonly used in scientific abstracts. We discuss what these results underline, namely that these discourse relations are based on particular phrasing that allow non-contextual encoders to perform well.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSentence EmbeddingsSimilar Papers 제목 키워드 기반
A Review of Discourse-level Machine Translation
Machine translation (MT) models usually translate a text at sentence level by considering isolated sentences, which is based on a strict assumption that the sentences in a text are independent of one another. However, th…
Machine TranslationNMTSentenceTranslationDisSent: Sentence Representation Learning from Explicit Discourse Relations
Learning effective representations of sentences is one of the core missions of natural language understanding. Existing models either train on a vast amount of text, or require costly, manually curated sentence relation …
Dependency ParsingNatural Language UnderstandingRelationRelation Prediction+3Recurrent Convolutional Neural Networks for Discourse Compositionality
The compositionality of meaning extends beyond the single sentence. Just as words combine to form the meaning of sentences, so do sentences combine to form the meaning of paragraphs, dialogues and general discourse. We i…
Dialogue Act ClassificationFeature EngineeringSentenceDisSent: Learning Sentence Representations from Explicit Discourse Relations
Learning effective representations of sentences is one of the core missions of natural language understanding. Existing models either train on a vast amount of text, or require costly, manually curated sentence relation …
Dependency ParsingNatural Language UnderstandingRelationRelation Prediction+2Correcting the Common Discourse Bias in Linear Representation of Sentences using Conceptors
Distributed representations of words, better known as word embeddings, have become important building blocks for natural language processing tasks. Numerous studies are devoted to transferring the success of unsupervised…
Semantic Textual SimilaritySentenceSentence EmbeddingSentence-Embedding+2