Regularizing and Aggregating Clients with Class Distribution for Personalized Federated Learning
Personalized federated learning (PFL) enables customized models for clients with varying data distributions. However, existing PFL methods often incur high computational and communication costs, limiting their practical application. This paper proposes a novel PFL method, Class-wise Federated Averaging (cwFedAVG), that performs Federated Averaging (FedAVG) class-wise, creating multiple global models per class on the server. Each local model integrates these global models weighted by its estimated local class distribution, derived from the L2-norms of deep network weights, avoiding privacy violations. Afterward, each global model does the same with local models using the same method. We also newly designed Weight Distribution Regularizer (WDR) to further enhance the accuracy of estimating a local class distribution by minimizing the Euclidean distance between the class distribution and the weight norms' distribution. Experimental results demonstrate that cwFedAVG matches or outperforms several existing PFL methods. Notably, cwFedAVG is conceptually simple yet computationally efficient as it mitigates the need for extensive calculation to collaborate between clients by leveraging shared global models. Visualizations provide insights into how cwFedAVG enables local model specialization on respective class distributions while global models capture class-relevant information across clients.
Code (1)
Tasks
Federated LearningPersonalized Federated LearningSimilar Papers 제목 키워드 기반
FLIS: Clustered Federated Learning via Inference Similarity for Non-IID Data Distribution
Classical federated learning approaches yield significant performance degradation in the presence of Non-IID data distributions of participants. When the distribution of each local dataset is highly different from the gl…
Federated LearningPersonalized Federated LearningpFedSim: Similarity-Aware Model Aggregation Towards Personalized Federated Learning
The federated learning (FL) paradigm emerges to preserve data privacy during model training by only exposing clients' model parameters rather than original data. One of the biggest challenges in FL lies in the non-IID (n…
Federated LearningPersonalized Federated LearningTask-Centric Personalized Federated Fine-Tuning of Language Models
Federated Learning (FL) has emerged as a promising technique for training language models on distributed and private datasets of diverse tasks. However, aggregating models trained on heterogeneous tasks often degrades th…
Federated LearningThe Diversity Bonus: Learning from Dissimilar Distributed Clients in Personalized Federated Learning
Personalized Federated Learning (PFL) is a commonly used framework that allows clients to collaboratively train their personalized models. PFL is particularly useful for handling situations where data from different clie…
DiversityFederated LearningPersonalized Federated LearningPersonalized Federated Learning With Graph
Knowledge sharing and model personalization are two key components in the conceptual framework of personalized federated learning (PFL). Existing PFL methods focus on proposing new model personalization mechanisms while …
Federated LearningPersonalized Federated LearningRelation