Forget but Recall: Incremental Latent Rectification in Continual Learning

2024-06-25 · Nghia D. Nguyen, Hieu Trung Nguyen, Ang Li, Hoang Pham, Viet Anh Nguyen, Khoa D. Doan

Intrinsic capability to continuously learn a changing data stream is a desideratum of deep neural networks (DNNs). However, current DNNs suffer from catastrophic forgetting, which hinders remembering past knowledge. To mitigate this issue, existing Continual Learning (CL) approaches either retain exemplars for replay, regularize learning, or allocate dedicated capacity for new tasks. This paper investigates an unexplored CL direction for incremental learning called Incremental Latent Rectification or ILR. In a nutshell, ILR learns to propagate with correction (or rectify) the representation from the current trained DNN backward to the representation space of the old task, where performing predictive decisions is easier. This rectification process only employs a chain of small representation mapping networks, called rectifier units. Empirical experiments on several continual learning benchmarks, including CIFAR10, CIFAR100, and Tiny ImageNet, demonstrate the effectiveness and potential of this novel CL direction compared to existing representative CL methods.

📄 PDF Abstract BibTeX arXiv:2406.17381

Code (0)

등록된 구현이 없습니다.

Tasks

Continual Learning Incremental Learning

Incremental Concept Learning via Online Generative Memory Recall

2019-07-05 · Huaiyu Li, Wei-Ming Dong, Bao-Gang Hu

The ability to learn more and more concepts over time from incrementally arriving data is essential for the development of a life-long learning system. However, deep neural networks often suffer from forgetting previousl…

class-incremental learning Class Incremental Learning Generative Adversarial Network Incremental Learning

Rectification-based Knowledge Retention for Continual Learning

2021-03-30 · CVPR 2021 1 · Pravendra Singh, Pratik Mazumder, Piyush Rai, Vinay P. Namboodiri

Deep learning models suffer from catastrophic forgetting when trained in an incremental learning setting. In this work, we propose a novel approach to address the task incremental learning problem, which involves trainin…

Continual Learning Generalized Zero-Shot Learning Incremental Learning Zero-Shot Learning

RECALL+: Adversarial Web-based Replay for Continual Learning in Semantic Segmentation

2023-09-19 · Chang Liu, Giulia Rizzoli, Francesco Barbato, Andrea Maracani 외

Catastrophic forgetting of previous knowledge is a critical issue in continual learning typically handled through various regularization strategies. However, existing methods struggle especially when several incremental …

Continual Learning Incremental Learning Semantic Segmentation

Latest Advancements Towards Catastrophic Forgetting under Data Scarcity: A Comprehensive Survey on Few-Shot Class Incremental Learning

2025-02-12 · M. Anwar Ma'sum, Mahardhika Pratama, Igor Skrjanc

Data scarcity significantly complicates the continual learning problem, i.e., how a deep neural network learns in dynamic environments with very few samples. However, the latest progress of few-shot class incremental lea…

class-incremental learning Class Incremental Learning Continual Learning Few-Shot Class-Incremental Learning+1

Compressed Latent Replays for Lightweight Continual Learning on Spiking Neural Networks

2024-05-08 · Alberto Dequino, Alessio Carpegna, Davide Nadalini, Alessandro Savino 외

Rehearsal-based Continual Learning (CL) has been intensely investigated in Deep Neural Networks (DNNs). However, its application in Spiking Neural Networks (SNNs) has not been explored in depth. In this paper we introduc…

Continual Learning