Fast Federated Learning in the Presence of Arbitrary Device Unavailability
Federated Learning (FL) coordinates with numerous heterogeneous devices to collaboratively train a shared model while preserving user privacy. Despite its multiple advantages, FL faces new challenges. One challenge arises when devices drop out of the training process beyond the control of the central server. In this case, the convergence of popular FL algorithms such as FedAvg is severely influenced by the straggling devices. To tackle this challenge, we study federated learning algorithms under arbitrary device unavailability and propose an algorithm named Memory-augmented Impatient Federated Averaging (MIFA). Our algorithm efficiently avoids excessive latency induced by inactive devices, and corrects the gradient bias using the memorized latest updates from the devices. We prove that MIFA achieves minimax optimal convergence rates on non-i.i.d. data for both strongly convex and non-convex smooth functions. We also provide an explicit characterization of the improvement over baseline algorithms through a case study, and validate the results by numerical experiments on real-world datasets.
Code (1)
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
Federated Learning in the Presence of Adversarial Client Unavailability
Federated learning is a decentralized machine learning framework that enables collaborative model training without revealing raw data. Due to the diverse hardware and software limitations, a client may not always be avai…
Federated LearningSelection biasEfficient Federated Learning against Heterogeneous and Non-stationary Client Unavailability
Addressing intermittent client availability is critical for the real-world deployment of federated learning algorithms. Most prior work either overlooks the potential non-stationarity in the dynamics of client unavailabi…
Federated LearningFedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling
We develop FedCluster--a novel federated learning framework with improved optimization efficiency, and investigate its theoretical convergence properties. The FedCluster groups the devices into multiple clusters that per…
Federated LearningFederated Learning with Erroneous Communication Links
In this paper, we consider the federated learning (FL) problem in the presence of communication errors. We model the link between the devices and the central node (CN) by a packet erasure channel, where the local paramet…
Federated LearningCorrFL: Correlation-based Neural Network Architecture for Unavailability Concerns in a Heterogeneous IoT Environment
The Federated Learning (FL) paradigm faces several challenges that limit its application in real-world environments. These challenges include the local models’ architecture heterogeneity and the unavailability of distrib…
Federated LearningRepresentation Learning