FedBABU: Toward Enhanced Representation for Federated Image Classification
Federated learning has evolved to improve a single global model under data heterogeneity (as a curse) or to develop multiple personalized models using data heterogeneity (as a blessing). However, little research has considered both directions simultaneously. In this paper, we first investigate the relationship between them by analyzing Federated Averaging at the client level and determine that a better federated global model performance does not constantly improve personalization. To elucidate the cause of this personalization performance degradation problem, we decompose the entire network into the body (i.e., extractor), which is related to universality, and the head (i.e., classifier), which is related to personalization. We then point out that this problem stems from training the head. Based on this observation, we propose a novel federated learning algorithm, coined FedBABU, which only updates the body of the model during federated training (i.e., the head is randomly initialized and never updated), and the head is fine-tuned for personalization during the evaluation process. Extensive experiments show consistent performance improvements and an efficient personalization of FedBABU. The code is available at https://anonymous.4open.science/r/ICLR2022-FedBABU.
Code (1)
Tasks
ClassificationFederated Learningimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
FedBABU: Towards Enhanced Representation for Federated Image Classification
Federated learning has evolved to improve a single global model under data heterogeneity (as a curse) or to develop multiple personalized models using data heterogeneity (as a blessing). However, little research has cons…
ClassificationFederated Learningimage-classificationImage ClassificationFedSPC: Shared Parameter Correction for Personalized Federated Learning
Personalized federated learning (PFL) is one of the important approaches in federated learning for addressing statistical heterogeneity while enabling client-specific adaptation. Many PFL methods split the model into sha…
Personalized Federated LearningClient Contribution Normalization for Enhanced Federated Learning
Mobile devices, including smartphones and laptops, generate decentralized and heterogeneous data, presenting significant challenges for traditional centralized machine learning models due to substantial communication cos…
Federated LearningFederated Distillation for Medical Image Classification: Towards Trustworthy Computer-Aided Diagnosis
Medical image classification plays a crucial role in computer-aided clinical diagnosis. While deep learning techniques have significantly enhanced efficiency and reduced costs, the privacy-sensitive nature of medical ima…
ClassificationFederated Learningimage-classificationImage Classification+2MQFL-FHE: Multimodal Quantum Federated Learning Framework with Fully Homomorphic Encryption
The integration of fully homomorphic encryption (FHE) in federated learning (FL) has led to significant advances in data privacy. However, during the aggregation phase, it often results in performance degradation of the …
Federated LearningMixture-of-Experts