Learn the Time to Learn: Replay Scheduling for Continual Learning
Replay-based continual learning has shown to be successful in mitigating catastrophic forgetting. Most previous works focus on increasing the sample quality in the commonly small replay memory. However, in many real-world applications, replay memories would be limited by constraints on processing time rather than storage capacity as most organizations do store all historical data in the cloud. Inspired by human learning, we illustrate that scheduling over which tasks to revisit is critical to the final performance with finite memory resources. To this end, we propose to learn the time to learn for a continual learning system, in which we learn schedules over which tasks to replay at different times using Monte Carlo tree search. We perform extensive evaluation and show that our method can learn replay schedules that significantly improve final performance across all tasks than baselines without considering the scheduling. Furthermore, our method can be combined with any other memory selection methods leading to consistently improved performance. Our results indicate that the learned schedules are also consistent with human learning insights.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningSchedulingSimilar Papers 제목 키워드 기반
Learn the Time to Learn: Replay Scheduling in Continual Learning
Replay methods are known to be successful at mitigating catastrophic forgetting in continual learning scenarios despite having limited access to historical data. However, storing historical data is cheap in many real-wor…
Continual LearningSchedulingWhen to Review: Spaced Repetition for Continual Pre-Training of Language Models
Continual pre-training of large language models must acquire new information without erasing old knowledge. Existing replay methods often choose a global old/new mixture and sample uniformly, ignoring that examples diffe…
Continual LearningMSSR: Memory-Aware Adaptive Replay for Continual LLM Fine-Tuning
Continual fine-tuning of large language models (LLMs) is becoming increasingly crucial as these models are deployed in dynamic environments where tasks and data distributions evolve over time. While strong adaptability e…
2nd Place Solution for SODA10M Challenge 2021 -- Continual Detection Track
In this technical report, we present our approaches for the continual object detection track of the SODA10M challenge. We adapt ResNet50-FPN as the baseline and try several improvements for the final submission model. We…
Autonomous Drivingobject-detectionObject DetectionPosition+2Closed-loop Control for Online Continual Learning
Online class-incremental continual learning (CL) deals with the sequential task learning problem in a realistic non-stationary setting with a single-pass through of data. Replay-based CL methods have shown promising resu…
Continual Learning