FedGradNorm: Personalized Federated Gradient-Normalized Multi-Task Learning
Multi-task learning (MTL) is a novel framework to learn several tasks simultaneously with a single shared network where each task has its distinct personalized header network for fine-tuning. MTL can be implemented in federated learning settings as well, in which tasks are distributed across clients. In federated settings, the statistical heterogeneity due to different task complexities and data heterogeneity due to non-iid nature of local datasets can both degrade the learning performance of the system. In addition, tasks can negatively affect each other's learning performance due to negative transference effects. To cope with these challenges, we propose FedGradNorm which uses a dynamic-weighting method to normalize gradient norms in order to balance learning speeds among different tasks. FedGradNorm improves the overall learning performance in a personalized federated learning setting. We provide convergence analysis for FedGradNorm by showing that it has an exponential convergence rate. We also conduct experiments on multi-task facial landmark (MTFL) and wireless communication system dataset (RadComDynamic). The experimental results show that our framework can achieve faster training performance compared to equal-weighting strategy. In addition to improving training speed, FedGradNorm also compensates for the imbalanced datasets among clients.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningMulti-Task LearningPersonalized Federated LearningSimilar Papers 제목 키워드 기반
Hierarchical Over-the-Air FedGradNorm
Multi-task learning (MTL) is a learning paradigm to learn multiple related tasks simultaneously with a single shared network where each task has a distinct personalized header network for fine-tuning. MTL can be integrat…
Federated LearningMulti-Task LearningPersonalized Federated LearningpFedSOP : Accelerating Training Of Personalized Federated Learning Using Second-Order Optimization
Personalized Federated Learning (PFL) enables clients to collaboratively train personalized models tailored to their individual objectives, addressing the challenge of model generalization in traditional Federated Learni…
Federated Learningimage-classificationImage ClassificationPersonalized Federated LearningPersonalized Heterogeneous Federated Learning with Gradient Similarity
In the conventional federated learning (FL), the local models of multiple clients are trained independently by their privacy data, and the center server generates the shared global model by aggregating local models. How…
Federated LearningSentinel: Dynamic Knowledge Distillation for Personalized Federated Intrusion Detection in Heterogeneous IoT Networks
Federated learning (FL) offers a privacy-preserving paradigm for machine learning, but its application in intrusion detection systems (IDS) within IoT networks is challenged by severe class imbalance, non-IID data, and h…
Knowledge DistillationIntrusion DetectionFederated LearningFedDAF: Federated Domain Adaptation Using Model Functional Distance
Federated Domain Adaptation (FDA) improves model performance at a target client by collaborating with source clients while preserving data privacy. FDA faces two key challenges: domain shift between source and target dat…
Federated LearningDomain Adaptation