Labeling Explicit Discourse Relations using Pre-trained Language Models
Labeling explicit discourse relations is one of the most challenging sub-tasks of the shallow discourse parsing where the goal is to identify the discourse connectives and the boundaries of their arguments. The state-of-the-art models achieve slightly above 45% of F-score by using hand-crafted features. The current paper investigates the efficacy of the pre-trained language models in this task. We find that the pre-trained language models, when finetuned, are powerful enough to replace the linguistic features. We evaluate our model on PDTB 2.0 and report the state-of-the-art results in the extraction of the full relation. This is the first time when a model outperforms the knowledge intensive models without employing any linguistic features.
Code (0)
등록된 구현이 없습니다.
Tasks
Discourse ParsingSimilar Papers 제목 키워드 기반
Argument Labeling of Explicit Discourse Relations using LSTM Neural Networks
Argument labeling of explicit discourse relations is a challenging task. The state of the art systems achieve slightly above 55% F-measure but require hand-crafted features. In this paper, we propose a Long Short Term Me…
Feature EngineeringAssessing Discourse Relations in Language Generation from GPT-2
Recent advances in NLP have been attributed to the emergence of large-scale pre-trained language models. GPT-2, in particular, is suited for generation tasks given its left-to-right language modeling objective, yet the l…
Language ModelingLanguage ModellingText GenerationvalidNeural Methodius Revisited: Do Discourse Relations Help with Pre-Trained Models Too?
Recent developments in natural language generation (NLG) have bolstered arguments in favor of re-introducing explicit coding of discourse relations in the input to neural models. In the Methodius corpus, a meaning repres…
RelationText GenerationTowards Identifying Alternative-Lexicalization Signals of Discourse Relations
The task of shallow discourse parsing in the Penn Discourse Treebank (PDTB) framework has traditionally been restricted to identifying those relations that are signaled by a discourse connective (“explicit”) and those th…
Discourse ParsingGenerating Discourse Connectives with Pre-trained Language Models: Conditioning on Discourse Relations Helps Reconstruct the PDTB
We report results of experiments using BART (Lewis et al., 2019) and the Penn Discourse Tree Bank (Webber et al., 2019) (PDTB) to generate texts with correctly realized discourse relations. We address a question left ope…
RelationText Generation