Making Document-Level Information Extraction Right for the Right Reasons
Document-level models for information extraction tasks like slot-filling are flexible: they can be applied to settings where information is not necessarily localized in a single sentence. For example, key features of a diagnosis in a radiology report may not be explicitly stated in one place, but nevertheless can be inferred from parts of the report's text. However, these models can easily learn spurious correlations between labels and irrelevant information. This work studies how to ensure that these models make correct inferences from complex text and make those inferences in an auditable way: beyond just being right, are these models "right for the right reasons?" We experiment with post-hoc evidence extraction in a predict-select-verify framework using feature attribution techniques. We show that regularization with small amounts of evidence supervision during training can substantially improve the quality of extracted evidence. We evaluate on two domains: a small-scale labeled dataset of brain MRI reports and a large-scale modified version of DocRED (Yao et al., 2019) and show that models' plausibility can be improved with no loss in accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Sentenceslot-fillingSlot FillingSimilar Papers 제목 키워드 기반
Making Document-Level Information Extraction Right for the Right Reasons
Document-level information extraction is a flexible framework compatible with applications where information is not necessarily localized in a single sentence. For example, key features of a diagnosis in radiology a repo…
SentencePRiSM: Enhancing Low-Resource Document-Level Relation Extraction with Relation-Aware Score Calibration
Document-level relation extraction (DocRE) aims to extract relations of all entity pairs in a document. A key challenge in DocRE is the cost of annotating such data which requires intensive human effort. Thus, we investi…
Document-level Relation ExtractionRelationRelation ExtractionDocument-level Relation Extraction with Dual-tier Heterogeneous Graph
Document-level relation extraction (RE) poses new challenges over its sentence-level counterpart since it requires an adequate comprehension of the whole document and the multi-hop reasoning ability across multiple sente…
Decision MakingDocument-level Relation ExtractionRelationRelation Extraction+1A Concept-Centered Hypertext Approach to Case-Based Retrieval
The goal of case-based retrieval is to assist physicians in the clinical decision making process, by finding relevant medical literature in large archives. We propose a research that aims at improving the effectiveness o…
Decision MakingRelational ReasoningRetrievalHarvesting Events from Multiple Sources: Towards a Cross-Document Event Extraction Paradigm
Document-level event extraction aims to extract structured event information from unstructured text. However, a single document often contains limited event information and the roles of different event arguments may be b…
coreference-resolutionCoreference ResolutionDocument-level Event ExtractionEvent Extraction