FedAH: Aggregated Head for Personalized Federated Learning
Recently, Federated Learning (FL) has gained popularity for its privacy-preserving and collaborative learning capabilities. Personalized Federated Learning (PFL), building upon FL, aims to address the issue of statistical heterogeneity and achieve personalization. Personalized-head-based PFL is a common and effective PFL method that splits the model into a feature extractor and a head, where the feature extractor is collaboratively trained and shared, while the head is locally trained and not shared. However, retaining the head locally, although achieving personalization, prevents the model from learning global knowledge in the head, thus affecting the performance of the personalized model. To solve this problem, we propose a novel PFL method called Federated Learning with Aggregated Head (FedAH), which initializes the head with an Aggregated Head at each iteration. The key feature of FedAH is to perform element-level aggregation between the local model head and the global model head to introduce global information from the global model head. To evaluate the effectiveness of FedAH, we conduct extensive experiments on five benchmark datasets in the fields of computer vision and natural language processing. FedAH outperforms ten state-of-the-art FL methods in terms of test accuracy by 2.87%. Additionally, FedAH maintains its advantage even in scenarios where some clients drop out unexpectedly. Our code is open-accessed at https://github.com/heyuepeng/FedAH.
Code (1)
Tasks
Federated LearningPersonalized Federated LearningPrivacy PreservingSimilar Papers 제목 키워드 기반
Dual Attention Heads for Personalized Federated Learning in ECG Classification
Federated learning (FL) enables collaborative model training across institutions without sharing sensitive patient data. However, the inherent heterogeneity of electrocardiogram (ECG) data across healthcare providers pre…
Personalized Federated LearningECG ClassificationPersonalized Federated Learning via Backbone Self-Distillation
In practical scenarios, federated learning frequently necessitates training personalized models for each client using heterogeneous data. This paper proposes a backbone self-distillation approach to facilitate personaliz…
Federated LearningPersonalized Federated LearningTransfer LearningPersonalized Federated Learning via Learning Dynamic Graphs
Personalized Federated Learning (PFL) aims to train a personalized model for each client that is tailored to its local data distribution, learning fails to perform well on individual clients due to variations in their lo…
Federated LearningGraph AttentionPersonalized Federated LearningPGFed: Personalize Each Client's Global Objective for Federated Learning
Personalized federated learning has received an upsurge of attention due to the mediocre performance of conventional federated learning (FL) over heterogeneous data. Unlike conventional FL which trains a single global co…
Federated LearningPersonalized Federated LearningTransfer LearningpFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning
Federated learning (FL) is an emerging machine learning paradigm in which a central server coordinates multiple participants (clients) collaboratively to train on decentralized data. In practice, FL often faces statistic…
Federated LearningPersonalized Federated Learning