TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questions
A critical part of reading is being able to understand the temporal relationships between events described in a passage of text, even when those relationships are not explicitly stated. However, current machine reading comprehension benchmarks have practically no questions that test temporal phenomena, so systems trained on these benchmarks have no capacity to answer questions such as "what happened before/after [some event]?" We introduce TORQUE, a new English reading comprehension benchmark built on 3.2k news snippets with 21k human-generated questions querying temporal relationships. Results show that RoBERTa-large achieves an exact-match score of 51% on the test set of TORQUE, about 30% behind human performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine Reading ComprehensionQuestion AnsweringReading ComprehensionSimilar Papers 제목 키워드 기반
Understand before Answer: Improve Temporal Reading Comprehension via Precise Question Understanding
This work studies temporal reading comprehension (TRC), which reads a free-text passage and answers temporal ordering questions. Precise question understanding is critical for temporal reading comprehension. For example,…
Reading ComprehensionRepresentation LearningUnderstand before Answer: Improve Temporal Reading Comprehension via Precise Question Understanding
This work studies temporal reading comprehension (TRC), which reads a free-text passage and answers temporal ordering questions. Precise question understanding is critical for temporal reading comprehension. For example,…
Reading ComprehensionRepresentation LearningOpen Temporal Relation Extraction for Question Answering
Understanding the temporal relations among events in text is a critical aspect of reading comprehension, which can be evaluated in the form of temporal question answering (TQA). When explicit timestamps are absent, TQA i…
Question AnsweringReading ComprehensionRelationRelation Extraction+1ESTER: A Machine Reading Comprehension Dataset for Event Semantic Relation Reasoning
Understanding how events are semantically related to each other is the essence of reading comprehension. Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. While th…
Machine Reading ComprehensionNatural Language QueriesQuestion AnsweringReading Comprehension+1ESTER: A Machine Reading Comprehension Dataset for Reasoning about Event Semantic Relations
Understanding how events are semantically related to each other is the essence of reading comprehension. Recent event-centric reading comprehension datasets focus mostly on event arguments or temporal relations. While th…
Machine Reading ComprehensionNatural Language QueriesReading ComprehensionRelation