Overcoming Catastrophic Forgetting in Federated Class-Incremental Learning via Federated Global Twin Generator
Federated Class-Incremental Learning (FCIL) increasingly becomes important in the decentralized setting, where it enables multiple participants to collaboratively train a global model to perform well on a sequence of tasks without sharing their private data. In FCIL, conventional Federated Learning algorithms such as FedAVG often suffer from catastrophic forgetting, resulting in significant performance declines on earlier tasks. Recent works, based on generative models, produce synthetic images to help mitigate this issue across all classes, but these approaches' testing accuracy on previous classes is still much lower than recent classes, i.e., having better plasticity than stability. To overcome these issues, this paper presents Federated Global Twin Generator (FedGTG), an FCIL framework that exploits privacy-preserving generative-model training on the global side without accessing client data. Specifically, the server trains a data generator and a feature generator to create two types of information from all seen classes, and then it sends the synthetic data to the client side. The clients then use feature-direction-controlling losses to make the local models retain knowledge and learn new tasks well. We extensively analyze the robustness of FedGTG on natural images, as well as its ability to converge to flat local minima and achieve better-predicting confidence (calibration). Experimental results on CIFAR-10, CIFAR-100, and tiny-ImageNet demonstrate the improvements in accuracy and forgetting measures of FedGTG compared to previous frameworks.
Code (0)
등록된 구현이 없습니다.
Tasks
class-incremental learningClass Incremental LearningFederated LearningIncremental LearningPrivacy PreservingSimilar Papers 제목 키워드 기반
An Efficient Replay for Class-Incremental Learning with Pre-trained Models
In general class-incremental learning, researchers typically use sample sets as a tool to avoid catastrophic forgetting during continuous learning. At the same time, researchers have also noted the differences between cl…
class-incremental learningClass Incremental LearningIncremental LearningCauses of Catastrophic Forgetting in Class-Incremental Semantic Segmentation
Class-incremental learning for semantic segmentation (CiSS) is presently a highly researched field which aims at updating a semantic segmentation model by sequentially learning new semantic classes. A major challenge in …
class-incremental learningClass Incremental LearningClass-Incremental Semantic SegmentationIncremental Learning+3Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima
This paper considers incremental few-shot learning, which requires a model to continually recognize new categories with only a few examples provided. Our study shows that existing methods severely suffer from catastrophi…
Few-Shot Class-Incremental LearningFew-Shot LearningIncremental LearningOvercoming Catastrophic Forgetting in Tabular Data Classification: A Pseudorehearsal-based approach
Continual learning (CL) poses the important challenge of adapting to evolving data distributions without forgetting previously acquired knowledge while consolidating new knowledge. In this paper, we introduce a new metho…
Continual LearningLifelong learningA Data-Free Approach to Mitigate Catastrophic Forgetting in Federated Class Incremental Learning for Vision Tasks
Deep learning models often suffer from forgetting previously learned information when trained on new data. This problem is exacerbated in federated learning (FL), where the data is distributed and can change independentl…
class-incremental learningClass Incremental LearningContinual LearningFederated Learning+1