Optimizing Personalized Federated Learning through Adaptive Layer-Wise Learning
Real-life deployment of federated Learning (FL) often faces non-IID data, which leads to poor accuracy and slow convergence. Personalized FL (pFL) tackles these issues by tailoring local models to individual data sources and using weighted aggregation methods for client-specific learning. However, existing pFL methods often fail to provide each local model with global knowledge on demand while maintaining low computational overhead. Additionally, local models tend to over-personalize their data during the training process, potentially dropping previously acquired global information. We propose FLAYER, a novel layer-wise learning method for pFL that optimizes local model personalization performance. FLAYER considers the different roles and learning abilities of neural network layers of individual local models. It incorporates global information for each local model as needed to initialize the local model cost-effectively. It then dynamically adjusts learning rates for each layer during local training, optimizing the personalized learning process for each local model while preserving global knowledge. Additionally, to enhance global representation in pFL, FLAYER selectively uploads parameters for global aggregation in a layer-wise manner. We evaluate FLAYER on four representative datasets in computer vision and natural language processing domains. Compared to six state-of-the-art pFL methods, FLAYER improves the inference accuracy, on average, by 7.21\% (up to 14.29\%).
Code (1)
Tasks
Federated LearningPersonalized Federated LearningSimilar Papers 제목 키워드 기반
pMixFed: Efficient Personalized Federated Learning through Adaptive Layer-Wise Mixup
Traditional Federated Learning (FL) methods encounter significant challenges when dealing with heterogeneous data and providing personalized solutions for non-IID scenarios. Personalized Federated Learning (PFL) approach…
Federated LearningPersonalized Federated LearningTowards Generalizable Personalized Federated Learning with Adaptive Local Adaptation
Personalized federated learning aims to find a shared global model that can be adapted to meet personal needs on each individual device. Starting from such a shared initial model, devices should be able to easily adapt t…
Federated LearningMeta-LearningPersonalized Federated LearningPersonalized Over-the-Air Federated Learning with Personalized Reconfigurable Intelligent Surfaces
Over-the-air federated learning (OTA-FL) provides bandwidth-efficient learning by leveraging the inherent superposition property of wireless channels. Personalized federated learning balances performance for users with d…
Federated LearningMulti-Task LearningPersonalized Federated LearningDynamic Personalized Federated Learning with Adaptive Differential Privacy
Personalized federated learning with differential privacy has been considered a feasible solution to address non-IID distribution of data and privacy leakage risks. However, current personalized federated learning method…
FedD2S: Personalized Data-Free Federated Knowledge Distillation
This paper addresses the challenge of mitigating data heterogeneity among clients within a Federated Learning (FL) framework. The model-drift issue, arising from the noniid nature of client data, often results in subopti…
Data-free Knowledge DistillationFairnessFederated LearningKnowledge Distillation+1