Relation Extraction with Weighted Contrastive Pre-training on Distant Supervision
Contrastive pre-training on distant supervision has shown remarkable effectiveness in improving supervised relation extraction tasks. However, the existing methods ignore the intrinsic noise of distant supervision during the pre-training stage. In this paper, we propose a weighted contrastive learning method by leveraging the supervised data to estimate the reliability of pre-training instances and explicitly reduce the effect of noise. Experimental results on three supervised datasets demonstrate the advantages of our proposed weighted contrastive learning approach compared to two state-of-the-art non-weighted baselines.Our code and models are available at: https://github.com/YukinoWan/WCL
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningRelationRelation ExtractionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
HiCLRE: A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction
Distant supervision assumes that any sentence containing the same entity pairs reflects identical relationships. Previous works of distantly supervised relation extraction (DSRE) task generally focus on sentence-level or…
Contrastive LearningData AugmentationRelationRelation Extraction+1Improving Distantly Supervised Relation Extraction using Word and Entity Based Attention
Relation extraction is the problem of classifying the relationship between two entities in a given sentence. Distant Supervision (DS) is a popular technique for developing relation extractors starting with limited superv…
RelationRelation ExtractionRelationship Extraction (Distant Supervised)SentenceA class-weighted supervised contrastive learning long-tailed bearing fault diagnosis approach using quadratic neural network
Deep learning has achieved remarkable success in bearing fault diagnosis. However, its performance oftentimes deteriorates when dealing with highly imbalanced or long-tailed data, while such cases are prevalent in indust…
Contrastive LearningData AugmentationFault DiagnosisCIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction
The journey of reducing noise from distant supervision (DS) generated training data has been started since the DS was first introduced into the relation extraction (RE) task. For the past decade, researchers apply the mu…
RelationRelation ExtractionSentenceFine-grained Contrastive Learning for Relation Extraction
Recent relation extraction (RE) works have shown encouraging improvements by conducting contrastive learning on silver labels generated by distant supervision before fine-tuning on gold labels. Existing methods typically…
Contrastive LearningDenoisingLanguage ModelingLanguage Modelling+2