Continual Lifelong Learning in Natural Language Processing: A Survey
Continual learning (CL) aims to enable information systems to learn from a continuous data stream across time. However, it is difficult for existing deep learning architectures to learn a new task without largely forgetting previously acquired knowledge. Furthermore, CL is particularly challenging for language learning, as natural language is ambiguous: it is discrete, compositional, and its meaning is context-dependent. In this work, we look at the problem of CL through the lens of various NLP tasks. Our survey discusses major challenges in CL and current methods applied in neural network models. We also provide a critical review of the existing CL evaluation methods and datasets in NLP. Finally, we present our outlook on future research directions.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningLifelong learningSurveySimilar Papers 제목 키워드 기반
Lifelong Learning of Large Language Model based Agents: A Roadmap
Lifelong learning, also known as continual or incremental learning, is a crucial component for advancing Artificial General Intelligence (AGI) by enabling systems to continuously adapt in dynamic environments. While larg…
Incremental LearningLanguage ModelingLanguage ModellingLarge Language Model+2Towards Lifelong Learning of Large Language Models: A Survey
As the applications of large language models (LLMs) expand across diverse fields, the ability of these models to adapt to ongoing changes in data, tasks, and user preferences becomes crucial. Traditional training methods…
Continual PretrainingIncremental LearningLifelong learningSurveyDRILL: Dynamic Representations for Imbalanced Lifelong Learning
Continual or lifelong learning has been a long-standing challenge in machine learning to date, especially in natural language processing (NLP). Although state-of-the-art language models such as BERT have ushered in a new…
Continual LearningLifelong learningMeta-Learningtext-classification+1How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition
A major goal of artificial intelligence (AI) is to create an agent capable of acquiring a general understanding of the world. Such an agent would require the ability to continually accumulate and build upon its knowledge…
Continual LearningLifelong learningLifelong and Continual Learning Dialogue Systems
Dialogue systems, commonly known as chatbots, have gained escalating popularity in recent times due to their wide-spread applications in carrying out chit-chat conversations with users and task-oriented dialogues to acco…
Continual LearningLifelong learning