Variance-Reduced Heterogeneous Federated Learning via Stratified Client Selection
Client selection strategies are widely adopted to handle the communication-efficient problem in recent studies of Federated Learning (FL). However, due to the large variance of the selected subset's update, prior selection approaches with a limited sampling ratio cannot perform well on convergence and accuracy in heterogeneous FL. To address this problem, in this paper, we propose a novel stratified client selection scheme to reduce the variance for the pursuit of better convergence and higher accuracy. Specifically, to mitigate the impact of heterogeneity, we develop stratification based on clients' local data distribution to derive approximate homogeneous strata for better selection in each stratum. Concentrating on a limited sampling ratio scenario, we next present an optimized sample size allocation scheme by considering the diversity of stratum's variability, with the promise of further variance reduction. Theoretically, we elaborate the explicit relation among different selection schemes with regard to variance, under heterogeneous settings, we demonstrate the effectiveness of our selection scheme. Experimental results confirm that our approach not only allows for better performance relative to state-of-the-art methods but also is compatible with prevalent FL algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityFederated LearningSimilar Papers 제목 키워드 기반
Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Framework
As a novel distributed learning paradigm, federated learning (FL) faces serious challenges in dealing with massive clients with heterogeneous data distribution and computation and communication resources. Various client-…
Federated Learningimage-classificationImage ClassificationSemi-Supervised Image ClassificationStratify: Rethinking Federated Learning for Non-IID Data through Balanced Sampling
Federated Learning (FL) on non-independently and identically distributed (non-IID) data remains a critical challenge, as existing approaches struggle with severe data heterogeneity. Current methods primarily address symp…
Federated LearningFaster Non-Convex Federated Learning via Global and Local Momentum
We propose \texttt{FedGLOMO}, a novel federated learning (FL) algorithm with an iteration complexity of $\mathcal{O}(\epsilon^{-1.5})$ to converge to an $\epsilon$-stationary point (i.e., $\mathbb{E}[\|\nabla f(\bm{x})\|…
Federated LearningFedOAED: Federated On-Device Autoencoder Denoiser for Heterogeneous Data under Limited Client Availability
Over the last few decades, machine learning (ML) and deep learning (DL) solutions have demonstrated their potential across many applications by leveraging large amounts of high-quality data. However, strict data-sharing …
Federated LearningSociodynamics-inspired Adaptive Coalition and Client Selection in Federated Learning
Federated Learning (FL) enables privacy-preserving collaborative model training, yet its practical strength is often undermined by client data heterogeneity, which severely degrades model performance. This paper proposes…
Federated LearningPrivacy Preserving