TEAL: New Selection Strategy for Small Buffers in Experience Replay Class Incremental Learning

2024-06-30 · Shahar Shaul-Ariel, Daphna Weinshall

Continual Learning is an unresolved challenge, whose relevance increases when considering modern applications. Unlike the human brain, trained deep neural networks suffer from a phenomenon called Catastrophic Forgetting, where they progressively lose previously acquired knowledge upon learning new tasks. To mitigate this problem, numerous methods have been developed, many relying on replaying past exemplars during new task training. However, as the memory allocated for replay decreases, the effectiveness of these approaches diminishes. On the other hand, maintaining a large memory for the purpose of replay is inefficient and often impractical. Here we introduce TEAL, a novel approach to populate the memory with exemplars, that can be integrated with various experience-replay methods and significantly enhance their performance on small memory buffers. We show that TEAL improves the average accuracy of the SOTA method XDER as well as ER and ER-ACE on several image recognition benchmarks, with a small memory buffer of 1-3 exemplars per class in the final task. This confirms the hypothesis that when memory is scarce, it is best to prioritize the most typical data.

📄 PDF Abstract BibTeX arXiv:2407.00673

Code (1)

shahariel/TEAL 공식 구현 pytorch

Tasks

class-incremental learning Class Incremental Learning Continual Learning Incremental Learning

Leveraging Complementary Embeddings for Replay Selection in Continual Learning with Small Buffers

2026-04-09 · Danit Yanowsky, Daphna Weinshall arxiv

Catastrophic forgetting remains a key challenge in Continual Learning (CL). In replay-based CL with severe memory constraints, performance critically depends on the sample selection strategy for the replay buffer. Most e…

Continual Learning

Heads collapse, features stay: Why Replay needs big buffers

2025-12-08 · Giulia Lanzillotta, Damiano Meier, Thomas Hofmann arxiv

A persistent paradox in continual learning (CL) is that neural networks often retain linearly separable representations of past tasks even when their output predictions fail. We formalize this distinction as the gap betw…

Out-of-Distribution Detection Continual Learning

When Classic Cache Policies Fail: Learning-Augmented Replacement for Semantic Retrieval Buffers

2026-07-01 · Yushi Sun, Bowen Cao, Wai Lam hf

LLM agents increasingly rely on retrieval buffers to store and reuse past experience, yet the cache management policies governing these buffers remain largely ad-hoc. We formalize this as an online semantic cache replace…

Semantic Retrieval

B2RL: An open-source Dataset for Building Batch Reinforcement Learning

2022-09-30 · Hsin-Yu Liu, Xiaohan Fu, Bharathan Balaji, Rajesh Gupta 외

Batch reinforcement learning (BRL) is an emerging research area in the RL community. It learns exclusively from static datasets (i.e. replay buffers) without interaction with the environment. In the offline settings, exi…

Management reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

ARROW: Augmented Replay for RObust World models

2026-03-12 · Abdulaziz Alyahya, Abdallah Al Siyabi, Markus R. Ernst, Luke Yang 외 arxiv

Continual reinforcement learning challenges agents to acquire new skills while retaining previously learned ones with the goal of improving performance in both past and future tasks. Most existing approaches rely on mode…

Reinforcement Learning