Prompting Implicit Discourse Relation Annotation
Pre-trained large language models, such as ChatGPT, archive outstanding performance in various reasoning tasks without supervised training and were found to have outperformed crowdsourcing workers. Nonetheless, ChatGPT's performance in the task of implicit discourse relation classification, prompted by a standard multiple-choice question, is still far from satisfactory and considerably inferior to state-of-the-art supervised approaches. This work investigates several proven prompting techniques to improve ChatGPT's recognition of discourse relations. In particular, we experimented with breaking down the classification task that involves numerous abstract labels into smaller subtasks. Nonetheless, experiment results show that the inference accuracy hardly changes even with sophisticated prompt engineering, suggesting that implicit discourse relation classification is not yet resolvable under zero-shot or few-shot settings.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationImplicit Discourse Relation ClassificationMultiple-choicePrompt EngineeringRelationRelation ClassificationSimilar Papers 제목 키워드 기반
Towards Full Text Shallow Discourse Relation Annotation: Experiments with Cross-Paragraph Implicit Relations in the PDTB
Full text discourse parsing relies on texts comprehensively annotated with discourse relations. To this end, we address a significant gap in the inter-sentential discourse relations annotated in the Penn Discourse Treeba…
Discourse ParsingImplicit RelationsDesign Choices for Crowdsourcing Implicit Discourse Relations: Revealing the Biases Introduced by Task Design
Disagreement in natural language annotation has mostly been studied from a perspective of biases introduced by the annotators and the annotation frameworks. Here, we propose to analyze another source of bias: task design…
TDB 1.1: Extensions on Turkish Discourse Bank
This paper presents the recent developments on Turkish Discourse Bank (TDB). First, the resource is summarized and an evaluation is presented. Then, TDB 1.1, i.e. enrichments on 10{\%} of the corpus are described (namely…
Implicit RelationsRelationAnnotation-Inspired Implicit Discourse Relation Classification with Auxiliary Discourse Connective Generation
Implicit discourse relation classification is a challenging task due to the absence of discourse connectives. To overcome this issue, we design an end-to-end neural model to explicitly generate discourse connectives for …
Implicit Discourse Relation ClassificationRelationRelation ClassificationImplicit Discourse Relation Classification via Multi-Task Neural Networks
Without discourse connectives, classifying implicit discourse relations is a challenging task and a bottleneck for building a practical discourse parser. Previous research usually makes use of one kind of discourse frame…
ClassificationGeneral ClassificationImplicit Discourse Relation ClassificationMulti-Task Learning+2