Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory

2023-07-02 · Sara Babakniya, Zalan Fabian, Chaoyang He, Mahdi Soltanolkotabi, Salman Avestimehr

Deep learning models are prone to forgetting information learned in the past when trained on new data. This problem becomes even more pronounced in the context of federated learning (FL), where data is decentralized and subject to independent changes for each user. Continual Learning (CL) studies this so-called \textit{catastrophic forgetting} phenomenon primarily in centralized settings, where the learner has direct access to the complete training dataset. However, applying CL techniques to FL is not straightforward due to privacy concerns and resource limitations. This paper presents a framework for federated class incremental learning that utilizes a generative model to synthesize samples from past distributions instead of storing part of past data. Then, clients can leverage the generative model to mitigate catastrophic forgetting locally. The generative model is trained on the server using data-free methods at the end of each task without requesting data from clients. Therefore, it reduces the risk of data leakage as opposed to training it on the client's private data. We demonstrate significant improvements for the CIFAR-100 dataset compared to existing baselines.

📄 PDF Abstract BibTeX arXiv:2307.00497

Code (0)

등록된 구현이 없습니다.

Tasks

class-incremental learning Class Incremental Learning Continual Learning Federated Learning Incremental Learning

Exemplar-condensed Federated Class-incremental Learning

2024-12-25 · Rui Sun, Yumin Zhang, Varun Ojha, Tejal Shah 외

We propose Exemplar-Condensed federated class-incremental learning (ECoral) to distil the training characteristics of real images from streaming data into informative rehearsal exemplars. The proposed method eliminates t…

class-incremental learning Class Incremental Learning Continual Learning Disentanglement+1

Remind of the Past: Incremental Learning with Analogical Prompts

2023-03-24 · Zhiheng Ma, Xiaopeng Hong, Beinan Liu, Yabin Wang 외

Although data-free incremental learning methods are memory-friendly, accurately estimating and counteracting representation shifts is challenging in the absence of historical data. This paper addresses this thorny proble…

Incremental Learning

A Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Tasks

2023-11-13 · NeurIPS 2023 11

Deep learning models often suffer from forgetting previously learned information when trained on new data. This problem is exacerbated in federated learning (FL), where the data is distributed and can change independentl…

class-incremental learning Class Incremental Learning Continual Learning Federated Learning+1

Bring Evanescent Representations to Life in Lifelong Class Incremental Learning

2022-01-01 · CVPR 2022 1 · Marco Toldo, Mete Ozay

In Class Incremental Learning (CIL), a classification model is progressively trained at each incremental step on an evolving dataset of new classes, while at the same time, it is required to preserve knowledge of all…

class-incremental learning Class Incremental Learning Exemplar-Free Incremental Learning

FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learning

2022-11-23 · Grégoire Petit, Adrian Popescu, Hugo Schindler, David Picard 외

Exemplar-free class-incremental learning is very challenging due to the negative effect of catastrophic forgetting. A balance between stability and plasticity of the incremental process is needed in order to obtain good …

class-incremental learning Class Incremental Learning Exemplar-Free Incremental Learning+1