FedMeS: Personalized Federated Continual Learning Leveraging Local Memory
We focus on the problem of Personalized Federated Continual Learning (PFCL): a group of distributed clients, each with a sequence of local tasks on arbitrary data distributions, collaborate through a central server to train a personalized model at each client, with the model expected to achieve good performance on all local tasks. We propose a novel PFCL framework called Federated Memory Strengthening FedMeS to address the challenges of client drift and catastrophic forgetting. In FedMeS, each client stores samples from previous tasks using a small amount of local memory, and leverages this information to both 1) calibrate gradient updates in training process; and 2) perform KNN-based Gaussian inference to facilitate personalization. FedMeS is designed to be task-oblivious, such that the same inference process is applied to samples from all tasks to achieve good performance. FedMeS is analyzed theoretically and evaluated experimentally. It is shown to outperform all baselines in average accuracy and forgetting rate, over various combinations of datasets, task distributions, and client numbers.
Code (0)
등록된 구현이 없습니다.
Tasks
AllContinual LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Personalized Federated Continual Learning via Multi-granularity Prompt
Personalized Federated Continual Learning (PFCL) is a new practical scenario that poses greater challenges in sharing and personalizing knowledge. PFCL not only relies on knowledge fusion for server aggregation at the gl…
Continual LearningFederated LearningPersonalized Federated LearningFedMes: Speeding Up Federated Learning with Multiple Edge Servers
We consider federated learning with multiple wireless edge servers having their own local coverages. We focus on speeding up training in this increasingly practical setup. Our key idea is to utilize the devices located i…
Federated LearningLarge-Small Model Collaborative Framework for Federated Continual Learning
Continual learning (CL) for Foundation Models (FMs) is an essential yet underexplored challenge, especially in Federated Continual Learning (FCL), where each client learns from a private, evolving task stream under stric…
Continual LearningBayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representations
Bayesian personalized federated learning (BPFL) addresses challenges in existing personalized FL (PFL). BPFL aims to quantify the uncertainty and heterogeneity within and across clients towards uncertainty representation…
Federated LearningPersonalized Federated LearningContinual Horizontal Federated Learning for Heterogeneous Data
Federated learning is a promising machine learning technique that enables multiple clients to collaboratively build a model without revealing the raw data to each other. Among various types of federated learning methods,…
Continual LearningFederated Learning