A Sample-Based Training Method for Distantly Supervised Relation Extraction with Pre-Trained Transformers
Multiple instance learning (MIL) has become the standard learning paradigm for distantly supervised relation extraction (DSRE). However, due to relation extraction being performed at bag level, MIL has significant hardware requirements for training when coupled with large sentence encoders such as deep transformer neural networks. In this paper, we propose a novel sampling method for DSRE that relaxes these hardware requirements. In the proposed method, we limit the number of sentences in a batch by randomly sampling sentences from the bags in the batch. However, this comes at the cost of losing valid sentences from bags. To alleviate the issues caused by random sampling, we use an ensemble of trained models for prediction. We demonstrate the effectiveness of our approach by using our proposed learning setting to fine-tuning BERT on the widely NYT dataset. Our approach significantly outperforms previous state-of-the-art methods in terms of AUC and P@N metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
Multiple Instance LearningRelationRelation ExtractionSentencevalidMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Distantly Supervised Relation Extraction with Self-Ensemble Noise Filtering
Distantly supervised models are very popular for relation extraction since we can obtain a large amount of training data using the distant supervision method without human annotation. In distant supervision, a sentence i…
RelationRelation ExtractionSentenceEmphasis on Easy Samples for Distantly Supervised Relation Extraction
There are many wrongly-labeled samples and low-quality samples in automatically generated Distantly Supervised Relation Extraction datasets. Overfitting these samples leads to decline of generalization. To address this i…
RelationRelation ExtractionSentenceRevisiting the Negative Data of Distantly Supervised Relation Extraction
Distantly supervision automatically generates plenty of training samples for relation extraction. However, it also incurs two major problems: noisy labels and imbalanced training data. Previous works focus more on reduci…
RelationRelation ExtractionSentenceKnowing False Negatives: An Adversarial Training Method for Distantly Supervised Relation Extraction
Distantly supervised relation extraction (RE) automatically aligns unstructured text with relation instances in a knowledge base (KB). Due to the incompleteness of current KBs, sentences implying certain relations may be…
RelationRelation ExtractionBootstrapping Distantly Supervised IE using Joint Learning and Small Well-structured Corpora
We propose a framework to improve performance of distantly-supervised relation extraction, by jointly learning to solve two related tasks: concept-instance extraction and relation extraction. We combine this with a novel…
RelationRelation Extraction