FedShift: Tackling Dual Heterogeneity Problem of Federated Learning via Weight Shift Aggregation
Federated Learning (FL) offers a compelling method for training machine learning models with a focus on preserving data privacy. The presence of system heterogeneity and statistical heterogeneity, recognized challenges in FL, arises from the diversity of client hardware, network, and dataset distribution. This diversity can critically affect the training pace and the performance of models. While many studies address either system or statistical heterogeneity by introducing communication-efficient or stable convergence algorithms, addressing these challenges in isolation often leads to compromises due to unaddressed heterogeneity. In response, this paper introduces FedShift, a novel algorithm designed to enhance both the training speed and the models' accuracy in a dual heterogeneity scenario. Our solution can improve client engagement through quantization and mitigate the adverse effects on performance typically associated with quantization by employing a shifting technique. This technique has proven to enhance accuracy by an average of 3.9% in diverse heterogeneity environments.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityFederated LearningQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Federated Learning with Classifier Shift for Class Imbalance
Federated learning aims to learn a global model collaboratively while the training data belongs to different clients and is not allowed to be exchanged. However, the statistical heterogeneity challenge on non-IID data, s…
Federated LearningTackling Computational Heterogeneity in FL: A Few Theoretical Insights
The future of machine learning lies in moving data collection along with training to the edge. Federated Learning, for short FL, has been recently proposed to achieve this goal. The principle of this approach is to aggre…
Federated LearningPreservation of the Global Knowledge by Not-True Distillation in Federated Learning
In federated learning, a strong global model is collaboratively learned by aggregating clients' locally trained models. Although this precludes the need to access clients' data directly, the global model's convergence of…
Continual LearningFederated LearningKnowledge DistillationSelf-Knowledge DistillationTackling Resource-Constrained and Data-Heterogeneity in Federated Learning with Double-Weight Sparse Pack
Federated learning has drawn widespread interest from researchers, yet the data heterogeneity across edge clients remains a key challenge, often degrading model performance. Existing methods enhance model compatibility w…
Personalized Federated LearningComputational EfficiencyKnowledge DistillationTackling Heterogeneity in Medical Federated learning via Vision Transformers
Optimization-based regularization methods have been effective in addressing the challenges posed by data heterogeneity in medical federated learning, particularly in improving the performance of underrepresented clients.…
Federated Learning