Personalized Federated Learning via Heterogeneous Modular Networks
Personalized Federated Learning (PFL) which collaboratively trains a federated model while considering local clients under privacy constraints has attracted much attention. Despite its popularity, it has been observed that existing PFL approaches result in sub-optimal solutions when the joint distribution among local clients diverges. To address this issue, we present Federated Modular Network (FedMN), a novel PFL approach that adaptively selects sub-modules from a module pool to assemble heterogeneous neural architectures for different clients. FedMN adopts a light-weighted routing hypernetwork to model the joint distribution on each client and produce the personalized selection of the module blocks for each client. To reduce the communication burden in existing FL, we develop an efficient way to interact between the clients and the server. We conduct extensive experiments on the real-world test beds and the results show both the effectiveness and efficiency of the proposed FedMN over the baselines.
Code (1)
Tasks
Federated LearningPersonalized Federated LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Communication-Efficient and Interoperable Distributed Learning
Collaborative learning across heterogeneous model architectures presents significant challenges in ensuring interoperability and preserving privacy. We propose a communication-efficient distributed learning framework tha…
Federated LearningpFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning
Federated learning (FL) has been widely adopted for collaborative training on decentralized data. However, it faces the challenges of data, system, and model heterogeneity. This has inspired the emergence of model-hetero…
Federated LearningMixture-of-ExpertsPersonalized Federated LearningFederated Neural Compression Under Heterogeneous Data
We discuss a federated learned compression problem, where the goal is to learn a compressor from real-world data which is scattered across clients and may be statistically heterogeneous, yet share a common underlying rep…
Federated LearningPersonalized Federated LearningFedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networks
Personalized federated learning is aimed at allowing numerous clients to train personalized models while participating in collaborative training in a communication-efficient manner without exchanging private data. Howeve…
Federated LearningPersonalized Federated LearningRepresentation LearningTransfer LearningHypernetworks for Model-Heterogeneous Personalized Federated Learning
Recent advances in personalized federated learning have focused on addressing client model heterogeneity. However, most existing methods still require external data, rely on model decoupling, or adopt partial learning st…
Personalized Federated Learning