Zero-shot Temporal Relation Extraction with ChatGPT
The goal of temporal relation extraction is to infer the temporal relation between two events in the document. Supervised models are dominant in this task. In this work, we investigate ChatGPT's ability on zero-shot temporal relation extraction. We designed three different prompt techniques to break down the task and evaluate ChatGPT. Our experiments show that ChatGPT's performance has a large gap with that of supervised methods and can heavily rely on the design of prompts. We further demonstrate that ChatGPT can infer more small relation classes correctly than supervised methods. The current shortcomings of ChatGPT on temporal relation extraction are also discussed in this paper. We found that ChatGPT cannot keep consistency during temporal inference and it fails in actively long-dependency temporal inference.
Code (0)
등록된 구현이 없습니다.
Tasks
RelationRelation ExtractionTemporal Relation ExtractionSimilar Papers 제목 키워드 기반
ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT
Zero-shot information extraction (IE) aims to build IE systems from the unannotated text. It is challenging due to involving little human intervention. Challenging but worthwhile, zero-shot IE reduces the time and effort…
Event Extractionnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1Consistency Guided Knowledge Retrieval and Denoising in LLMs for Zero-shot Document-level Relation Triplet Extraction
Document-level Relation Triplet Extraction (DocRTE) is a fundamental task in information systems that aims to simultaneously extract entities with semantic relations from a document. Existing methods heavily rely on a su…
DenoisingRelationRetrievalText Generation+1ChatGPT Evaluation on Sentence Level Relations: A Focus on Temporal, Causal, and Discourse Relations
This paper aims to quantitatively evaluate the performance of ChatGPT, an interactive large language model, on inter-sentential relations such as temporal relations, causal relations, and discourse relations. Given ChatG…
Discourse ParsingIn-Context LearningLanguage ModelingLanguage Modelling+5Large Language Models as Zero-Shot Keyphrase Extractors: A Preliminary Empirical Study
Zero-shot keyphrase extraction aims to build a keyphrase extractor without training by human-annotated data, which is challenging due to the limited human intervention involved. Challenging but worthwhile, zero-shot sett…
Keyphrase ExtractionLanguage ModelingLanguage ModellingLarge Language ModelRevisiting Large Language Models as Zero-shot Relation Extractors
Relation extraction (RE) consistently involves a certain degree of labeled or unlabeled data even if under zero-shot setting. Recent studies have shown that large language models (LLMs) transfer well to new tasks out-of-…
Question AnsweringRelationRelation Extraction