Towards Personalized Federated Learning via Heterogeneous Model Reassembly
This paper focuses on addressing the practical yet challenging problem of model heterogeneity in federated learning, where clients possess models with different network structures. To track this problem, we propose a novel framework called pFedHR, which leverages heterogeneous model reassembly to achieve personalized federated learning. In particular, we approach the problem of heterogeneous model personalization as a model-matching optimization task on the server side. Moreover, pFedHR automatically and dynamically generates informative and diverse personalized candidates with minimal human intervention. Furthermore, our proposed heterogeneous model reassembly technique mitigates the adverse impact introduced by using public data with different distributions from the client data to a certain extent. Experimental results demonstrate that pFedHR outperforms baselines on three datasets under both IID and Non-IID settings. Additionally, pFedHR effectively reduces the adverse impact of using different public data and dynamically generates diverse personalized models in an automated manner.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning
Federated learning (FL) has been widely adopted for collaborative training on decentralized data. However, it faces the challenges of data, system, and model heterogeneity. This has inspired the emergence of model-hetero…
Federated LearningMixture-of-ExpertsPersonalized Federated LearningFederated Neural Compression Under Heterogeneous Data
We discuss a federated learned compression problem, where the goal is to learn a compressor from real-world data which is scattered across clients and may be statistically heterogeneous, yet share a common underlying rep…
Federated LearningPersonalized Federated LearningFedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networks
Personalized federated learning is aimed at allowing numerous clients to train personalized models while participating in collaborative training in a communication-efficient manner without exchanging private data. Howeve…
Federated LearningPersonalized Federated LearningRepresentation LearningTransfer LearningHypernetworks for Model-Heterogeneous Personalized Federated Learning
Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning st…
Personalized Federated LearningFlexible Personalized Split Federated Learning for On-Device Fine-Tuning of Foundation Models
Fine-tuning foundation models is critical for superior performance on personalized downstream tasks, compared to using pre-trained models. Collaborative learning can leverage local clients' datasets for fine-tuning, but …
Personalized Federated Learning