Federated Learning with Partial Model Personalization
We consider two federated learning algorithms for training partially personalized models, where the shared and personal parameters are updated either simultaneously or alternately on the devices. Both algorithms have been proposed in the literature, but their convergence properties are not fully understood, especially for the alternating variant. We provide convergence analyses of both algorithms in the general nonconvex setting with partial participation and delineate the regime where one dominates the other. Our experiments on real-world image, text, and speech datasets demonstrate that (a) partial personalization can obtain most of the benefits of full model personalization with a small fraction of personal parameters, and, (b) the alternating update algorithm often outperforms the simultaneous update algorithm by a small but consistent margin.
Code (2)
Tasks
Federated LearningmodelSimilar Papers 제목 키워드 기반
FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learning
Personalized Federated Learning (PFL) represents a promising solution for decentralized learning in heterogeneous data environments. Partial model personalization has been proposed to improve the efficiency of PFL by sel…
Federated LearningPersonalized Federated LearningConnecting Low-Loss Subspace for Personalized Federated Learning
Due to the curse of statistical heterogeneity across clients, adopting a personalized federated learning method has become an essential choice for the successful deployment of federated learning-based services. Among div…
Ensemble LearningFederated LearningPersonalized Federated LearningFederated Reconstruction: Partially Local Federated Learning
Personalization methods in federated learning aim to balance the benefits of federated and local training for data availability, communication cost, and robustness to client heterogeneity. Approaches that require clients…
Collaborative FilteringFederated LearningMeta-LearningA Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning
Federated Learning (FL) enables decentralized, privacy-preserving model training but struggles to balance global generalization and local personalization due to non-identical data distributions across clients. Personaliz…
Federated LearningMAP: Model Aggregation and Personalization in Federated Learning with Incomplete Classes
In some real-world applications, data samples are usually distributed on local devices, where federated learning (FL) techniques are proposed to coordinate decentralized clients without directly sharing users' private da…
Federated Learning