Better Generative Replay for Continual Federated Learning
Federated learning is a technique that enables a centralized server to learn from distributed clients via communications without accessing the client local data. However, existing federated learning works mainly focus on a single task scenario with static data. In this paper, we introduce the problem of continual federated learning, where clients incrementally learn new tasks and history data cannot be stored due to certain reasons, such as limited storage and data retention policy. Generative replay based methods are effective for continual learning without storing history data, but adapting them for this setting is challenging. By analyzing the behaviors of clients during training, we find that the unstable training process caused by distributed training on non-IID data leads to a notable performance degradation. To address this problem, we propose our FedCIL model with two simple but effective solutions: model consolidation and consistency enforcement. Our experimental results on multiple benchmark datasets demonstrate that our method significantly outperforms baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningFederated LearningSimilar Papers 제목 키워드 기반
Coordinated Replay Sample Selection for Continual Federated Learning
Continual Federated Learning (CFL) combines Federated Learning (FL), the decentralized learning of a central model on a number of client devices that may not communicate their data, and Continual Learning (CL), the learn…
Continual LearningFederated LearningUsing Diffusion Models as Generative Replay in Continual Federated Learning -- What will Happen?
Federated learning (FL) has become a cornerstone in decentralized learning, where, in many scenarios, the incoming data distribution will change dynamically over time, introducing continuous learning (CL) problems. This …
Federated LearningTask-Agnostic Federated Continual Learning via Replay-Free Gradient Projection
Federated continual learning (FCL) enables collaborative model training across distributed clients on sequentially arriving tasks without revisiting past data. However, existing approaches often suffer from catastrophic …
Continual LearningGenerative Negative Replay for Continual Learning
Learning continually is a key aspect of intelligence and a necessary ability to solve many real-life problems. One of the most effective strategies to control catastrophic forgetting, the Achilles' heel of continual lear…
Continual LearningFeature Aggregation with Latent Generative Replay for Federated Continual Learning of Socially Appropriate Robot Behaviours
It is critical for robots to explore Federated Learning (FL) settings where several robots, deployed in parallel, can learn independently while also sharing their learning with each other. This collaborative learning in …
Continual LearningCPUFederated LearningGPU