Discourse Coherence in the Wild: A Dataset, Evaluation and Methods
To date there has been very little work on assessing discourse coherence methods on real-world data. To address this, we present a new corpus of real-world texts (GCDC) as well as the first large-scale evaluation of leading discourse coherence algorithms. We show that neural models, including two that we introduce here (SentAvg and ParSeq), tend to perform best. We analyze these performance differences and discuss patterns we observed in low coherence texts in four domains.
Code (1)
Tasks
Coherence EvaluationSimilar Papers 제목 키워드 기반
Unlocking Structure Measuring: Introducing PDD, an Automatic Metric for Positional Discourse Coherence
Recent large language models (LLMs) have shown remarkable performance in aligning generated text with user intentions across various tasks. When it comes to long-form text generation, there has been a growing interest in…
ArticlesCoherence EvaluationFormText GenerationJoint Modeling of Entities and Discourse Relations for Coherence Assessment
In linguistics, coherence can be achieved by different means, such as by maintaining reference to the same set of entities across sentences and by establishing discourse relations between them. However, most existing wor…
Coherence EvaluationDiscoScore: Evaluating Text Generation with BERT and Discourse Coherence
Recently, there has been a growing interest in designing text generation systems from a discourse coherence perspective, e.g., modeling the interdependence between sentences. Still, recent BERT-based evaluation metrics a…
Document Level Machine TranslationMachine TranslationText GenerationNeural Net Models of Open-domain Discourse Coherence
Discourse coherence is strongly associated with text quality, making it important to natural language generation and understanding. Yet existing models of coherence focus on measuring individual aspects of coherence (lex…
Abstractive Text SummarizationQuestion AnsweringSentence EmbeddingsText GenerationNeural Net Models for Open-Domain Discourse Coherence
Discourse coherence is strongly associated with text quality, making it important to natural language generation and understanding. Yet existing models of coherence focus on measuring individual aspects of coherence (lex…
Text Generation