Topic-Guided Coherence Modeling for Sentence Ordering by Preserving Global and Local Information
We propose a novel topic-guided coherence modeling (TGCM) for sentence ordering. Our attention based pointer decoder directly utilize sentence vectors in a permutation-invariant manner, without being compressed into a single fixed-length vector as the paragraph representation. Thus, TGCM can improve global dependencies among sentences and preserve relatively informative paragraph-level semantics. Moreover, to predict the next sentence, we capture topic-enhanced sentence-pair interactions between the current predicted sentence and each next-sentence candidate. With the coherent topical context matching, we promote local dependencies that help identify the tight semantic connections for sentence ordering. The experimental results show that TGCM outperforms state-of-the-art models from various perspectives.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderSentenceSentence OrderingSimilar Papers 제목 키워드 기반
Ordering-sensitive and Semantic-aware Topic Modeling
Topic modeling of textual corpora is an important and challenging problem. In most previous work, the "bag-of-words" assumption is usually made which ignores the ordering of words. This assumption simplifies the computat…
RetrievalEvaluating Text Coherence at Sentence and Paragraph Levels
In this paper, to evaluate text coherence, we propose the paragraph ordering task as well as conducting sentence ordering. We collected four distinct corpora from different domains on which we investigate the adaptation …
Graph Neural NetworkSentenceSentence OrderingAssessing the Coherence Modeling Capabilities of Pretrained Transformer-based Language Models
The task of ordering a shuffled set of sentences into a coherent text is used to evaluate the capacity of a model to understand causal and temporal relations between entities and events. Recent approaches rely on pretrai…
DiversitySentenceSentence OrderingTowards Document-Level Paraphrase Generation with Sentence Rewriting and Reordering
Paraphrase generation is an important task in natural language processing. Previous works focus on sentence-level paraphrase generation, while ignoring document-level paraphrase generation, which is a more challenging an…
DiversityParaphrase GenerationSentenceSentence ReWritingTopic Compositional Neural Language Model
We propose a Topic Compositional Neural Language Model (TCNLM), a novel method designed to simultaneously capture both the global semantic meaning and the local word ordering structure in a document. The TCNLM learns the…
Language ModelingLanguage ModellingMixture-of-Expertsmodel