MAML-CL: Edited Model-Agnostic Meta-Learning for Continual Learning
Continual learning (CL) exhibits a learning ability to well-learn all sequentially seen tasks drawn from various domains. Yet, existing sequential training methods fail to consolidate learned knowledge from earlier tasks due to data distribution shifts, hereby leading to catastrophic forgetting. We devise an optimization-based meta learning framework for CL in accordance with MAML, where query samples are edited for generalization of learned knowledge. We conduct extensive experiments on text classification in a low resource CL setup, where we downsize training set to its 10%. The experimental results demonstrate the superiority of our method in terms of stability, fast adaptation, memory efficiency and knowledge retention across various domains.
Code (1)
Tasks
Continual LearningMeta-Learningtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning
In recent years, model-agnostic meta-learning (MAML) has become a popular research area. However, the stochastic optimization of MAML is still underdeveloped. Existing MAML algorithms rely on the ``episode'' idea by samp…
Continual LearningFederated LearningMeta-LearningPersonalized Federated Learning+1Meta-Learning for Natural Language Understanding under Continual Learning Framework
Neural network has been recognized with its accomplishments on tackling various natural language understanding (NLU) tasks. Methods have been developed to train a robust model to handle multiple tasks to gain a general r…
Continual LearningMeta-LearningNatural Language UnderstandingLa-MAML: Look-ahead Meta Learning for Continual Learning
The continual learning problem involves training models with limited capacity to perform well on a set of an unknown number of sequentially arriving tasks. While meta-learning shows great potential for reducing interfere…
Continual LearningMeta-LearningIs Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably?
Meta learning aims at learning a model that can quickly adapt to unseen tasks. Widely used meta learning methods include model agnostic meta learning (MAML), implicit MAML, Bayesian MAML. Thanks to its ability of modelin…
Meta-LearningmodelOnline Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning
Continual learning studies agents that learn from streams of tasks without forgetting previous ones while adapting to new ones. Two recent continual-learning scenarios have opened new avenues of research. In meta-continu…
Continual LearningMeta-Learning