Less is More: Rethinking State-of-the-art Continual Relation Extraction Models with a Frustratingly Easy but Effective Approach
Continual relation extraction (CRE) requires the model to continually learn new relations from class-incremental data streams. In this paper, we propose a Frustratingly easy but Effective Approach (FEA) method with two learning stages for CRE: 1) Fast Adaption (FA) warms up the model with only new data. 2) Balanced Tuning (BT) finetunes the model on the balanced memory data. Despite its simplicity, FEA achieves comparable (on TACRED or superior (on FewRel) performance compared with the state-of-the-art baselines. With careful examinations, we find that the data imbalance between new and old relations leads to a skewed decision boundary in the head classifiers over the pretrained encoders, thus hurting the overall performance. In FEA, the FA stage unleashes the potential of memory data for the subsequent finetuning, while the BT stage helps establish a more balanced decision boundary. With a unified view, we find that two strong CRE baselines can be subsumed into the proposed training pipeline. The success of FEA also provides actionable insights and suggestions for future model designing in CRE.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual Relation ExtractionRelation ExtractionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Rethinking Continual Learning for Autonomous Agents and Robots
Continual learning refers to the ability of a biological or artificial system to seamlessly learn from continuous streams of information while preventing catastrophic forgetting, i.e., a condition in which new incoming i…
Continual LearningTransfer LearningContinual Learning Without Knowing Task Identities: Rethinking Occam's Razor
Due to the catastrophic forgetting phenomenon of deep neural networks (DNNs), models trained in standard ways tend to forget what it has learned from previous tasks, especially when the new task is sufficiently different…
Continual LearningModel SelectionRethinking the Foundations for Continual Reinforcement Learning
Algorithms and approaches for continual reinforcement learning have gained increasing attention. Much of this early progress rests on the foundations and standard practices of traditional reinforcement learning, without …
Continual Learningreinforcement-learningReinforcement LearningJust a Glimpse: Rethinking Temporal Information for Video Continual Learning
Class-incremental learning is one of the most important settings for the study of Continual Learning, as it closely resembles real-world application scenarios. With constrained memory sizes, catastrophic forgetting arise…
class-incremental learningClass Incremental LearningContinual LearningDiversity+1Deep Learning with a Rethinking Structure for Multi-label Classification
Multi-label classification (MLC) is an important class of machine learning problems that come with a wide spectrum of applications, each demanding a possibly different evaluation criterion. When solving the MLC problems,…
ClassificationDeep LearningGeneral ClassificationMulti-Label Classification+1