Continual Learning for Encoder-only Language Models via a Discrete Key-Value Bottleneck
Continual learning remains challenging across various natural language understanding tasks. When models are updated with new training data, they risk catastrophic forgetting of prior knowledge. In the present work, we introduce a discrete key-value bottleneck for encoder-only language models, allowing for efficient continual learning by requiring only localized updates. Inspired by the success of a discrete key-value bottleneck in vision, we address new and NLP-specific challenges. We experiment with different bottleneck architectures to find the most suitable variants regarding language, and present a generic discrete key initialization technique for NLP that is task independent. We evaluate the discrete key-value bottleneck in four continual learning NLP scenarios and demonstrate that it alleviates catastrophic forgetting. We showcase that it offers competitive performance to other popular continual learning methods, with lower computational costs.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningNatural Language UnderstandingSimilar Papers 제목 키워드 기반
Discrete Key-Value Bottleneck
Deep neural networks perform well on classification tasks where data streams are i.i.d. and labeled data is abundant. Challenges emerge with non-stationary training data streams such as continual learning. One powerful a…
class-incremental learningClass Incremental LearningContinual LearningIncremental LearningOnline Learned Continual Compression with Adaptive Quantization Modules
We introduce and study the problem of Online Continual Compression, where one attempts to simultaneously learn to compress and store a representative dataset from a non i.i.d data stream, while only observing each sample…
Continual LearningDecoderQuantizationReinforcement LearningAn Empirical Analysis of Discrete Unit Representations in Speech Language Modeling Pre-training
This paper investigates discrete unit representations in Speech Language Models (SLMs), focusing on optimizing speech modeling during continual pre-training. In this paper, we systematically examine how model architectur…
Sculpting Subspaces: Constrained Full Fine-Tuning in LLMs for Continual Learning
Continual learning in large language models (LLMs) is prone to catastrophic forgetting, where adapting to new tasks significantly degrades performance on previously learned ones. Existing methods typically rely on low-ra…
Continual LearningDecoderInstruction FollowingHarnessing Discrete Representations For Continual Reinforcement Learning
Reinforcement learning (RL) agents make decisions using nothing but observations from the environment, and consequently, heavily rely on the representations of those observations. Though some recent breakthroughs have us…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)