Active Continual Learning: On Balancing Knowledge Retention and Learnability
Acquiring new knowledge without forgetting what has been learned in a sequence of tasks is the central focus of continual learning (CL). While tasks arrive sequentially, the training data are often prepared and annotated independently, leading to the CL of incoming supervised learning tasks. This paper considers the under-explored problem of active continual learning (ACL) for a sequence of active learning (AL) tasks, where each incoming task includes a pool of unlabelled data and an annotation budget. We investigate the effectiveness and interplay between several AL and CL algorithms in the domain, class and task-incremental scenarios. Our experiments reveal the trade-off between two contrasting goals of not forgetting the old knowledge and the ability to quickly learn new knowledge in CL and AL, respectively. While conditioning the AL query strategy on the annotations collected for the previous tasks leads to improved task performance on the domain and task incremental learning, our proposed forgetting-learning profile suggests a gap in balancing the effect of AL and CL for the class-incremental scenario.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningContinual LearningIncremental LearningSimilar Papers 제목 키워드 기반
MacVQA: Adaptive Memory Allocation and Global Noise Filtering for Continual Visual Question Answering
Visual Question Answering (VQA) requires models to reason over multimodal information, combining visual and textual data. With the development of continual learning, significant progress has been made in retaining knowle…
Visual Question AnsweringContinual LearningBalancing Continual Learning and Fine-tuning for Human Activity Recognition
Wearable-based Human Activity Recognition (HAR) is a key task in human-centric machine learning due to its fundamental understanding of human behaviours. Due to the dynamic nature of human behaviours, continual learning …
Activity RecognitionContinual LearningContinual Self-Supervised LearningContrastive Learning+3Balanced Gradient Sample Retrieval for Enhanced Knowledge Retention in Proxy-based Continual Learning
Continual learning in deep neural networks often suffers from catastrophic forgetting, where representations for previous tasks are overwritten during subsequent training. We propose a novel sample retrieval strategy fro…
Continual LearningContrastive LearningDiversityRetrievalKeepLoRA++: Continual Learning with Layer-Scaled Residual Gradient Adaptation
Continual learning for pre-trained vision-language models requires balancing three competing objectives: retaining pre-trained knowledge, preserving knowledge from a sequence of learned tasks, and maintaining the plastic…
Visual Question AnsweringImage ClassificationContinual LearningBeyond Retention: Orchestrating Structural Safety and Plasticity in Continual Learning for LLMs
Continual learning in Large Language Models (LLMs) faces the critical challenge of balancing stability (retaining old knowledge) and plasticity (learning new tasks). While Experience Replay (ER) is a standard countermeas…
Continual LearningCode Generation