A Language Model-based Generative Classifier for Sentence-level Discourse Parsing
Discourse segmentation and sentence-level discourse parsing play important roles for various NLP tasks to consider textual coherence. Despite recent achievements in both tasks, there is still room for improvement due to the scarcity of labeled data. To solve the problem, we propose a language model-based generative classifier (LMGC) for using more information from labels by treating the labels as an input while enhancing label representations by embedding descriptions for each label. Moreover, since this enables LMGC to make ready the representations for labels, unseen in the pre-training step, we can effectively use a pre-trained language model in LMGC. Experimental results on the RST-DT dataset show that our LMGC achieved the state-of-the-art F1 score of 96.72 in discourse segmentation. It further achieved the state-of-the-art relation F1 scores of 84.69 with gold EDU boundaries and 81.18 with automatically segmented boundaries, respectively, in sentence-level discourse parsing.
Code (0)
등록된 구현이 없습니다.
Tasks
Discourse ParsingDiscourse SegmentationLanguage ModelingLanguage ModellingSegmentationSentenceSimilar Papers 제목 키워드 기반
An Annotated Dataset and Automatic Approaches for Discourse Mode Identification in Low-resource Bengali Language
The modes of discourse aid in comprehending the convention and purpose of various forms of languages used during communication. In this study, we introduce a discourse mode annotated corpus for the low-resource Bangla (a…
DescriptiveLanguage ModelingLanguage ModellingSentenceNeural Net Models of Open-domain Discourse Coherence
Discourse coherence is strongly associated with text quality, making it important to natural language generation and understanding. Yet existing models of coherence focus on measuring individual aspects of coherence (lex…
Abstractive Text SummarizationQuestion AnsweringSentence EmbeddingsText GenerationNeural Net Models for Open-Domain Discourse Coherence
Discourse coherence is strongly associated with text quality, making it important to natural language generation and understanding. Yet existing models of coherence focus on measuring individual aspects of coherence (lex…
Text GenerationDEPTH: Discourse Education through Pre-Training Hierarchically
Language Models (LMs) struggle with linguistic understanding at the discourse level, even though discourse patterns such as coherence, cohesion, and narrative flow are prevalent in their pre-training data. To improve the…
DecoderNatural Language UnderstandingSentenceAugmenting BERT-style Models with Predictive Coding to Improve Discourse-level Representations
Current language models are usually trained using a self-supervised scheme, where the main focus is learning representations at the word or sentence level. However, there has been limited progress in generating useful di…
Relationship DetectionSentence