I2I: Initializing Adapters with Improvised Knowledge
Adapters present a promising solution to the catastrophic forgetting problem in continual learning. However, training independent Adapter modules for every new task misses an opportunity for cross-task knowledge transfer. We propose Improvise to Initialize (I2I), a continual learning algorithm that initializes Adapters for incoming tasks by distilling knowledge from previously-learned tasks' Adapters. We evaluate I2I on CLiMB, a multimodal continual learning benchmark, by conducting experiments on sequences of visual question answering tasks. Adapters trained with I2I consistently achieve better task accuracy than independently-trained Adapters, demonstrating that our algorithm facilitates knowledge transfer between task Adapters. I2I also results in better cross-task knowledge transfer than the state-of-the-art AdapterFusion without incurring the associated parametric cost.
Code (1)
Tasks
Continual LearningQuestion AnsweringTransfer LearningVisual Question AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Linked Adapters: Linking Past and Future to Present for Effective Continual Learning
Continual learning allows the system to learn and adapt to new tasks while retaining the knowledge acquired from previous tasks. However, deep learning models suffer from catastrophic forgetting of knowledge learned from…
Continual Learningimage-classificationImage ClassificationTransfer LearningActed vs. Improvised: Domain Adaptation for Elicitation Approaches in Audio-Visual Emotion Recognition
Key challenges in developing generalized automatic emotion recognition systems include scarcity of labeled data and lack of gold-standard references. Even for the cues that are labeled as the same emotion category, the v…
Domain AdaptationEmotion RecognitionTransfer LearningAuto-selected Knowledge Adapters for Lifelong Person Re-identification
Lifelong Person Re-Identification (LReID) extends traditional ReID by requiring systems to continually learn from non-overlapping datasets across different times and locations, adapting to new identities while preserving…
Lifelong learningPerson Re-IdentificationHarnessing Diffusion-Yielded Score Priors for Image Restoration
Deep image restoration models aim to learn a mapping from degraded image space to natural image space. However, they face several critical challenges: removing degradation, generating realistic details, and ensuring pixe…
Image RestorationOn The Cross-Modal Transfer from Natural Language to Code through Adapter Modules
Pre-trained neural Language Models (PTLM), such as CodeBERT, are recently used in software engineering as models pre-trained on large source code corpora. Their knowledge is transferred to downstream tasks (e.g. code clo…
Clone DetectionCloze TestTransfer Learning