Improving Continual Relation Extraction through Prototypical Contrastive Learning
Continual relation extraction (CRE) aims to extract relations towards the continuous and iterative arrival of new data, of which the major challenge is the catastrophic forgetting of old tasks. In order to alleviate this critical problem for enhanced CRE performance, we propose a novel Continual Relation Extraction framework with Contrastive Learning, namely CRECL, which is built with a classification network and a prototypical contrastive network to achieve the incremental-class learning of CRE. Specifically, in the contrastive network a given instance is contrasted with the prototype of each candidate relations stored in the memory module. Such contrastive learning scheme ensures the data distributions of all tasks more distinguishable, so as to alleviate the catastrophic forgetting further. Our experiment results not only demonstrate our CRECL's advantage over the state-of-the-art baselines on two public datasets, but also verify the effectiveness of CRECL's contrastive learning on improving CRE performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual Relation ExtractionContrastive LearningRelationRelation ExtractionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Prototypical Representation Learning for Low-resource Knowledge Extraction: Summary and Perspective
Recent years have witnessed the success of prototypical representation in widespread low-resource tasks, since "Prototypical Networks for Few-shot Learning (NeurIPS 2017)" proposed to represent each class as a prototype …
Contrastive LearningFew-Shot LearningRelation ExtractionRepresentation LearningSerial Contrastive Knowledge Distillation for Continual Few-shot Relation Extraction
Continual few-shot relation extraction (RE) aims to continuously train a model for new relations with few labeled training data, of which the major challenges are the catastrophic forgetting of old relations and the over…
Contrastive LearningKnowledge DistillationRelationRelation Extraction+1Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation
Contrastive self-supervised learning (CSL) with a prototypical regularization has been introduced in learning meaningful representations for downstream tasks that require strong semantic information. However, to optimize…
Contrastive LearningDiversitySelf-Supervised LearningSteering Prototypes with Prompt-tuning for Rehearsal-free Continual Learning
In the context of continual learning, prototypes-as representative class embeddings-offer advantages in memory conservation and the mitigation of catastrophic forgetting. However, challenges related to semantic drift and…
class-incremental learningClass Incremental LearningContinual LearningContrastive Learning+1Consistent Representation Learning for Continual Relation Extraction
Continual relation extraction (CRE) aims to continuously train a model on data with new relations while avoiding forgetting old ones. Some previous work has proved that storing a few typical samples of old relations and …
Continual Relation ExtractionContrastive LearningKnowledge DistillationRelation+2