Generative Binary Memory: Pseudo-Replay Class-Incremental Learning on Binarized Embeddings

2025-03-13 · Yanis Basso-Bert, Anca Molnos, Romain Lemaire, William Guicquero, Antoine Dupret

In dynamic environments where new concepts continuously emerge, Deep Neural Networks (DNNs) must adapt by learning new classes while retaining previously acquired ones. This challenge is addressed by Class-Incremental Learning (CIL). This paper introduces Generative Binary Memory (GBM), a novel CIL pseudo-replay approach which generates synthetic binary pseudo-exemplars. Relying on Bernoulli Mixture Models (BMMs), GBM effectively models the multi-modal characteristics of class distributions, in a latent, binary space. With a specifically-designed feature binarizer, our approach applies to any conventional DNN. GBM also natively supports Binary Neural Networks (BNNs) for highly-constrained model sizes in embedded systems. The experimental results demonstrate that GBM achieves higher than state-of-the-art average accuracy on CIFAR100 (+2.9%) and TinyImageNet (+1.5%) for a ResNet-18 equipped with our binarizer. GBM also outperforms emerging CIL methods for BNNs, with +3.1% in final accuracy and x4.7 memory reduction, on CORE50.

📄 PDF Abstract BibTeX arXiv:2503.10333

Code (0)

등록된 구현이 없습니다.

Tasks

class-incremental learning Class Incremental Learning Incremental Learning

Class-Incremental Learning Using Generative Experience Replay Based on Time-aware Regularization

2023-10-05 · Zizhao Hu, Mohammad Rostami

Learning new tasks accumulatively without forgetting remains a critical challenge in continual learning. Generative experience replay addresses this challenge by synthesizing pseudo-data points for past learned tasks and…

class-incremental learning Class Incremental Learning Continual Learning Incremental Learning

BinPlay: A Binary Latent Autoencoder for Generative Replay Continual Learning

2020-11-25 · Kamil Deja, Paweł Wawrzyński, Daniel Marczak, Wojciech Masarczyk 외

We introduce a binary latent space autoencoder architecture to rehearse training samples for the continual learning of neural networks. The ability to extend the knowledge of a model with new data without forgetting prev…

Continual Learning

Generative Memory for Lifelong Reinforcement Learning

2019-02-22 · Aswin Raghavan, Jesse Hostetler, Sek Chai

Our research is focused on understanding and applying biological memory transfers to new AI systems that can fundamentally improve their performance, throughout their fielded lifetime experience. We leverage current unde…

Lifelong learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Replay to Remember (R2R): An Efficient Uncertainty-driven Unsupervised Continual Learning Framework Using Generative Replay

2025-05-07 · Sriram Mandalika, Harsha Vardhan, Athira Nambiar

Continual Learning entails progressively acquiring knowledge from new data while retaining previously acquired knowledge, thereby mitigating ``Catastrophic Forgetting'' in neural networks. Our work presents a novel uncer…

Continual Learning

Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay

2022-01-09 · Kuluhan Binici, Shivam Aggarwal, Nam Trung Pham, Karianto Leman 외

Data-Free Knowledge Distillation (KD) allows knowledge transfer from a trained neural network (teacher) to a more compact one (student) in the absence of original training data. Existing works use a validation set to mon…

Data-free Knowledge Distillation image-classification Image Classification Knowledge Distillation+1