Personalized Federated Learning through Global Knowledge Distillation and Local Head Adaptation
Statistical heterogeneity limits federated learning when a single global classifier cannot represent client-specific label distributions. In this work, we propose Personalized Federated Knowledge Distillation with Head Adaptation (pFedKDH), which aggregates only the shared backbone, keeps persistent client-specific heads, and uses a recalibrated global head as a teacher during local training. Across MNIST, Fashion-MNIST, CIFAR10, and CIFAR100 under class-wise Dirichlet partitions, pFedKDH obtains the best accuracy in most settings, with accuracy gaps up to 37.67\% over the weakest baseline and consistently low standard deviation across repetitions. Component-wise diagnostics and convergence results support the role of persistent heads and distillation-guided local optimization under label-skewed data.
Code (0)
등록된 구현이 없습니다.
Tasks
Personalized Federated LearningKnowledge DistillationSimilar Papers 제목 키워드 기반
Towards Personalized Federated Learning via Comprehensive Knowledge Distillation
Federated learning is a distributed machine learning paradigm designed to protect data privacy. However, data heterogeneity across various clients results in catastrophic forgetting, where the model rapidly forgets previ…
Federated LearningKnowledge DistillationPersonalized Federated LearningThe Best of Both Worlds: Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillation
Heterogeneity of data distributed across clients limits the performance of global models trained through federated learning, especially in the settings with highly imbalanced class distributions of local datasets. In rec…
Federated LearningKnowledge DistillationPersonalized Federated LearningPersonalized Federated Learning via Backbone Self-Distillation
In practical scenarios, federated learning frequently necessitates training personalized models for each client using heterogeneous data. This paper proposes a backbone self-distillation approach to facilitate personaliz…
Federated LearningPersonalized Federated LearningTransfer LearningQuPeD: Quantized Personalization via Distillation with Applications to Federated Learning
Traditionally, federated learning (FL) aims to train a single global model while collaboratively using multiple clients and a server. Two natural challenges that FL algorithms face are heterogeneity in data across client…
Federated LearningKnowledge DistillationModel CompressionQuantizationFedD2S: Personalized Data-Free Federated Knowledge Distillation
This paper addresses the challenge of mitigating data heterogeneity among clients within a Federated Learning (FL) framework. The model-drift issue, arising from the noniid nature of client data, often results in subopti…
Data-free Knowledge DistillationFairnessFederated LearningKnowledge Distillation+1