What do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification
Over the last five years, research on Relation Extraction (RE) witnessed extensive progress with many new dataset releases. At the same time, setup clarity has decreased, contributing to increased difficulty of reliable empirical evaluation (Taill\'e et al., 2020). In this paper, we provide a comprehensive survey of RE datasets, and revisit the task definition and its adoption by the community. We find that cross-dataset and cross-domain setups are particularly lacking. We present an empirical study on scientific Relation Classification across two datasets. Despite large data overlap, our analysis reveals substantial discrepancies in annotation. Annotation discrepancies strongly impact Relation Classification performance, explaining large drops in cross-dataset evaluations. Variation within further sub-domains exists but impacts Relation Classification only to limited degrees. Overall, our study calls for more rigour in reporting setups in RE and evaluation across multiple test sets.
Code (1)
Tasks
ClassificationRelationRelation ClassificationRelation ExtractionSimilar Papers 제목 키워드 기반
Deep Neural Approaches to Relation Triplets Extraction: A Comprehensive Survey
Recently, with the advances made in continuous representation of words (word embeddings) and deep neural architectures, many research works are published in the area of relation extraction and it is very difficult to kee…
Document-level Relation ExtractionRelationRelation ExtractionSentence+2Techniques for Jointly Extracting Entities and Relations: A Survey
Relation Extraction is an important task in Information Extraction which deals with identifying semantic relations between entity mentions. Traditionally, relation extraction is carried out after entity extraction in a "…
RelationRelation ExtractionSurveyRelation Extraction : A Survey
With the advent of the Internet, large amount of digital text is generated everyday in the form of news articles, research publications, blogs, question answering forums and social media. It is important to develop techn…
ArticlesInformation RetrievalManagementOpen Information Extraction+6A Survey of Deep Learning Methods for Relation Extraction
Relation Extraction is an important sub-task of Information Extraction which has the potential of employing deep learning (DL) models with the creation of large datasets using distant supervision. In this review, we comp…
Deep LearningRelationRelation ExtractionSurveySurvey of Abstract Meaning Representation: Then, Now, Future
This paper presents a survey of Abstract Meaning Representation (AMR), a semantic representation framework that captures the meaning of sentences through a graph-based structure. AMR represents sentences as rooted, direc…
Abstract Meaning RepresentationSurveytext-classificationText Classification+1