HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association
Federated learning (FL) is a promising paradigm that enables collaboratively learning a shared model across massive clients while keeping the training data locally. However, for many existing FL systems, clients need to frequently exchange model parameters of large data size with the remote cloud server directly via wide-area networks (WAN), leading to significant communication overhead and long transmission time. To mitigate the communication bottleneck, we resort to the hierarchical federated learning paradigm of HiFL, which reaps the benefits of mobile edge computing and combines synchronous client-edge model aggregation and asynchronous edge-cloud model aggregation together to greatly reduce the traffic volumes of WAN transmissions. Specifically, we first analyze the convergence bound of HiFL theoretically and identify the key controllable factors for model performance improvement. We then advocate an enhanced design of HiFlash by innovatively integrating deep reinforcement learning based adaptive staleness control and heterogeneity-aware client-edge association strategy to boost the system efficiency and mitigate the staleness effect without compromising model accuracy. Extensive experiments corroborate the superior performance of HiFlash in model accuracy, communication reduction, and system efficiency.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Reinforcement LearningEdge-computingFederated LearningSimilar Papers 제목 키워드 기반
Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data
Federated learning (FL) is an emerging privacy-preserving paradigm that enables multiple participants collaboratively to train a global model without uploading raw data. Considering heterogeneous computing and communicat…
Federated LearningPrivacy PreservingAsynchronous Hierarchical Federated Learning
Federated Learning is a rapidly growing area of research and with various benefits and industry applications. Typical federated patterns have some intrinsic issues such as heavy server traffic, long periods of convergenc…
Federated Learningimage-classificationImage ClassificationMAB-Based Channel Scheduling for Asynchronous Federated Learning in Non-Stationary Environments
Federated learning enables distributed model training across clients under central coordination without raw data exchange. However, in wireless implementations, frequent parameter updates between the server and clients c…
FairnessFederated LearningSchedulingFederated Stochastic Gradient Descent Begets Self-Induced Momentum
Federated learning (FL) is an emerging machine learning method that can be applied in mobile edge systems, in which a server and a host of clients collaboratively train a statistical model utilizing the data and computat…
Federated LearningRevisiting Gradient Staleness: Evaluating Distance Metrics for Asynchronous Federated Learning Aggregation
In asynchronous federated learning (FL), client devices send updates to a central server at varying times based on their computational speed, often using stale versions of the global model. This staleness can degrade the…
Federated Learning