Optimal Importance Sampling for Federated Learning
Federated learning involves a mixture of centralized and decentralized processing tasks, where a server regularly selects a sample of the agents and these in turn sample their local data to compute stochastic gradients for their learning updates. This process runs continually. The sampling of both agents and data is generally uniform; however, in this work we consider non-uniform sampling. We derive optimal importance sampling strategies for both agent and data selection and show that non-uniform sampling without replacement improves the performance of the original FedAvg algorithm. We run experiments on a regression and classification problem to illustrate the theoretical results.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningregressionSimilar Papers 제목 키워드 기반
Federated Learning under Importance Sampling
Federated learning encapsulates distributed learning strategies that are managed by a central unit. Since it relies on using a selected number of agents at each iteration, and since each agent, in turn, taps into its loc…
Federated LearningISFL: Federated Learning for Non-i.i.d. Data with Local Importance Sampling
As a promising learning paradigm integrating computation and communication, federated learning (FL) proceeds the local training and the periodic sharing from distributed clients. Due to the non-i.i.d. data distribution o…
DiversityFederated LearningFederated Graph Learning with Adaptive Importance-based Sampling
For privacy-preserving graph learning tasks involving distributed graph datasets, federated learning (FL)-based GCN (FedGCN) training is required. A key challenge for FedGCN is scaling to large-scale graphs, which typica…
Federated LearningGraph LearningGraph SamplingPrivacy PreservingOptimal Client Sampling for Federated Learning
It is well understood that client-master communication can be a primary bottleneck in Federated Learning. In this work, we address this issue with a novel client subsampling scheme, where we restrict the number of client…
Federated LearningAdaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedback
Due to the high cost of communication, federated learning (FL) systems need to sample a subset of clients that are involved in each round of training. As a result, client sampling plays an important role in FL systems as…
Federated LearningStochastic Optimization