Pre-training Entity Relation Encoder with Intra-span and Inter-span Information
In this paper, we integrate span-related information into pre-trained encoder for entity relation extraction task. Instead of using general-purpose sentence encoder (e.g., existing universal pre-trained models), we introduce a span encoder and a span pair encoder to the pre-training network, which makes it easier to import intra-span and inter-span information into the pre-trained model. To learn the encoders, we devise three customized pre-training objectives from different perspectives, which target on tokens, spans, and span pairs. In particular, a span encoder is trained to recover a random shuffling of tokens in a span, and a span pair encoder is trained to predict positive pairs that are from the same sentences and negative pairs that are from different sentences using contrastive loss. Experimental results show that the proposed pre-training method outperforms distantly supervised pre-training, and achieves promising performance on two entity relation extraction benchmark datasets (ACE05, SciERC).
Code (0)
등록된 구현이 없습니다.
Tasks
RelationRelation ExtractionSentenceSimilar Papers 제목 키워드 기반
GapDNER: A Gap-Aware Grid Tagging Model for Discontinuous Named Entity Recognition
In biomedical fields, one named entity may consist of a series of non-adjacent tokens and overlap with other entities. Previous methods recognize discontinuous entities by connecting entity fragments or internal tokens, …
Representation LearningPacked Levitated Marker for Entity and Relation Extraction
Recent entity and relation extraction works focus on investigating how to obtain a better span representation from the pre-trained encoder. However, a major limitation of existing works is that they ignore the interrelat…
Joint Entity and Relation ExtractionNamed Entity Recognition (NER)RelationRelation ExtractionPacked Levitated Marker for Entity and Relation Extraction
Recent entity and relation extraction works focus on investigating how to obtain a better span representation from the pre-trained encoder. However, a major limitation of existing works is that they ignore the interrelat…
NERRelationRelation ExtractionMulti-Grained Query-Guided Set Prediction Network for Grounded Multimodal Named Entity Recognition
Grounded Multimodal Named Entity Recognition (GMNER) is an emerging information extraction (IE) task, aiming to simultaneously extract entity spans, types, and corresponding visual regions of entities from given sentence…
Grounded Multimodal Named Entity RecognitionMachine Reading Comprehensionnamed-entity-recognitionNamed Entity Recognition+2An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extraction
In this paper, we propose a novel method for joint entity and relation extraction from unstructured text by framing it as a conditional sequence generation problem. In contrast to conventional generative information extr…
DecoderJoint Entity and Relation ExtractionRelationRelation Extraction