paper-with-me

홈 › Papers

Forgetting Order of Continual Learning: Examples That are Learned First are Forgotten Last

2024-06-14 · Guy Hacohen, Tinne Tuytelaars

Catastrophic forgetting poses a significant challenge in continual learning, where models often forget previous tasks when trained on new data. Our empirical analysis reveals a strong correlation between catastrophic forgetting and the learning speed of examples: examples learned early are rarely forgotten, while those learned later are more susceptible to forgetting. We demonstrate that replay-based continual learning methods can leverage this phenomenon by focusing on mid-learned examples for rehearsal. We introduce Goldilocks, a novel replay buffer sampling method that filters out examples learned too quickly or too slowly, keeping those learned at an intermediate speed. Goldilocks improves existing continual learning algorithms, leading to state-of-the-art performance across several image classification tasks.

📄 PDF Abstract BibTeX arXiv:2406.09935

Code (0)

등록된 구현이 없습니다.

Tasks

Continual Learningimage-classificationImage Classification

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Similar Papers 제목 키워드 기반

GopGAN: Gradients Orthogonal Projection Generative Adversarial Network With Continual Learning

2021-07-16 · IEEE Transactions on Neural Networks and Learning Systems 2021 7 · Xiaobin Li; Weiqiang Wang

The generative adversarial networks (GANs) in continual learning suffer from catastrophic forgetting. In continual learning, GANs tend to forget about previous generation tasks and only remember the tasks they just learn…

Continual LearningGenerative Adversarial Network

Memory Population in Continual Learning via Outlier Elimination

2022-07-04 · Julio Hurtado, Alain Raymond-Saez, Vladimir Araujo, Vincenzo Lomonaco 외

Catastrophic forgetting, the phenomenon of forgetting previously learned tasks when learning a new one, is a major hurdle in developing continual learning algorithms. A popular method to alleviate forgetting is to use a …

Continual Learning

Preserving Earlier Knowledge in Continual Learning with the Help of All Previous Feature Extractors

2021-04-28 · Zhuoyun Li, Changhong Zhong, Sijia Liu, Ruixuan Wang 외

Continual learning of new knowledge over time is one desirable capability for intelligent systems to recognize more and more classes of objects. Without or with very limited amount of old data stored, an intelligent syst…

AllContinual Learning

Condensed Composite Memory Continual Learning

2021-02-19 · Felix Wiewel, Bin Yang

Deep Neural Networks (DNNs) suffer from a rapid decrease in performance when trained on a sequence of tasks where only data of the most recent task is available. This phenomenon, known as catastrophic forgetting, prevent…

Continual Learning

Gradient-based Editing of Memory Examples for Online Task-free Continual Learning

2020-06-27 · NeurIPS 2021 12 · Xisen Jin, Arka Sadhu, Junyi Du, Xiang Ren

We explore task-free continual learning (CL), in which a model is trained to avoid catastrophic forgetting in the absence of explicit task boundaries or identities. Among many efforts on task-free CL, a notable family of…

Continual Learning