Replay-free Online Continual Learning with Self-Supervised MultiPatches
Online Continual Learning (OCL) methods train a model on a non-stationary data stream where only a few examples are available at a time, often leveraging replay strategies. However, usage of replay is sometimes forbidden, especially in applications with strict privacy regulations. Therefore, we propose Continual MultiPatches (CMP), an effective plug-in for existing OCL self-supervised learning strategies that avoids the use of replay samples. CMP generates multiple patches from a single example and projects them into a shared feature space, where patches coming from the same example are pushed together without collapsing into a single point. CMP surpasses replay and other SSL-based strategies on OCL streams, challenging the role of replay as a go-to solution for self-supervised OCL.
Code (1)
Tasks
Continual LearningSelf-Supervised LearningSimilar Papers 제목 키워드 기반
CLIMB: Centroid-Based Hierarchical Memory for Online Continual Self-Supervised Learning
Online Continual Self-Supervised Learning (OCSSL) aims to learn representations from a continuous stream of unlabeled data, without knowledge of task boundaries and under memory constraints. Existing methods rely either …
Self-Supervised LearningKnowledge DistillationContrastive LearningContinual LearningLeveraging Complementary Embeddings for Replay Selection in Continual Learning with Small Buffers
Catastrophic forgetting remains a key challenge in Continual Learning (CL). In replay-based CL with severe memory constraints, performance critically depends on the sample selection strategy for the replay buffer. Most e…
Continual LearningContinual Learning on Noisy Data Streams via Self-Purified Replay
Continually learning in the real world must overcome many challenges, among which noisy labels are a common and inevitable issue. In this work, we present a repla-ybased continual learning framework that simultaneously a…
Continual LearningSelf-Supervised LearningDomain-Aware Augmentations for Unsupervised Online General Continual Learning
Continual Learning has been challenging, especially when dealing with unsupervised scenarios such as Unsupervised Online General Continual Learning (UOGCL), where the learning agent has no prior knowledge of class bounda…
Continual LearningContrastive LearningPreventing Latent Rehearsal Decay in Online Continual SSL with SOLAR
This paper explores Online Continual Self-Supervised Learning (OCSSL), a scenario in which models learn from continuous streams of unlabeled, non-stationary data, where methods typically employ replay and fast convergenc…
Self-Supervised Learning