Analysing Errors of Open Information Extraction Systems
We report results on benchmarking Open Information Extraction (OIE) systems using RelVis, a toolkit for benchmarking Open Information Extraction systems. Our comprehensive benchmark contains three data sets from the news domain and one data set from Wikipedia with overall 4522 labeled sentences and 11243 binary or n-ary OIE relations. In our analysis on these data sets we compared the performance of four popular OIE systems, ClausIE, OpenIE 4.2, Stanford OpenIE and PredPatt. In addition, we evaluated the impact of five common error classes on a subset of 749 n-ary tuples. From our deep analysis we unreveal important research directions for a next generation of OIE systems.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingOpen Information ExtractionSimilar Papers 제목 키워드 기반
Open Information Extraction on Scientific Text: An Evaluation
Open Information Extraction (OIE) is the task of the unsupervised creation of structured information from text. OIE is often used as a starting point for a number of downstream tasks including knowledge base construction…
Knowledge Base ConstructionOpen Information ExtractionQuestion AnsweringRelation ExtractionIntegrating Local Context and Global Cohesiveness for Open Information Extraction
Extracting entities and their relations from text is an important task for understanding massive text corpora. Open information extraction (IE) systems mine relation tuples (i.e., entity arguments and a predicate string …
Open Information ExtractionRelationSentence$\textit{BenchIE}^{FL}$ : A Manually Re-Annotated Fact-Based Open Information Extraction Benchmark
Open Information Extraction (OIE) is a field of natural language processing that aims to present textual information in a format that allows it to be organized, analyzed and reflected upon. Numerous OIE systems are devel…
Open Information ExtractionOn the Impact of Word Error Rate on Acoustic-Linguistic Speech Emotion Recognition: An Update for the Deep Learning Era
Text encodings from automatic speech recognition (ASR) transcripts and audio representations have shown promise in speech emotion recognition (SER) ever since. Yet, it is challenging to explain the effect of each informa…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Emotion RecognitionSelf-Supervised Learning+3In Layman's Terms: Semi-Open Relation Extraction from Scientific Texts
Information Extraction (IE) from scientific texts can be used to guide readers to the central information in scientific documents. But narrow IE systems extract only a fraction of the information captured, and Open IE sy…
RelationRelation Extraction