Diverse Client Selection for Federated Learning via Submodular Maximization
In every communication round of federated learning, a random subset of clients communicate their model updates back to the server which then aggregates them all. The optimal size of this subset is not known and several studies have shown that typically random selection does not perform very well in terms of convergence, learning efficiency and fairness. We, in this paper, propose to select a small diverse subset of clients, namely those carrying representative gradient information, and we transmit only these updates to the server. Our aim is for updating via only a subset to approximate updating via aggregating all client information. We achieve this by choosing a subset that maximizes a submodular facility location function defined over gradient space. We introduce “federated averaging with diverse client selection (DivFL)”. We provide a thorough analysis of its convergence in the heterogeneous setting and apply it both to synthetic and to real datasets. Empirical results show several benefits to our approach including improved learning efficiency, faster convergence and also more uniform (i.e., fair) performance across clients. We further show a communication-efficient version of DivFL that can still outperform baselines on the above metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
FairnessFederated LearningSimilar Papers 제목 키워드 기반
RobustFSM: Submodular Maximization in Federated Setting with Malicious Clients
Submodular maximization is an optimization problem benefiting many machine learning applications, where we seek a small subset best representing an extremely large dataset. We focus on the federated setting where the dat…
Federated LearningSubmodular Maximization Approaches for Equitable Client Selection in Federated Learning
In a conventional Federated Learning framework, client selection for training typically involves the random sampling of a subset of clients in each iteration. However, this random selection often leads to disparate perfo…
FairnessFederated LearningDecomposable Submodular Maximization in Federated Setting
Submodular functions, as well as the sub-class of decomposable submodular functions, and their optimization appear in a wide range of applications in machine learning, recommendation systems, and welfare maximization. Ho…
Recommendation SystemsAdaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning
Federated learning (FL) allows multiple clients cooperatively train models without disclosing local data. However, the existing works fail to address all these practical concerns in FL: limited communication resources, d…
Federated LearningFederated Learning with Fair Worker Selection: A Multi-Round Submodular Maximization Approach
In this paper, we study the problem of fair worker selection in Federated Learning systems, where fairness serves as an incentive mechanism that encourages more workers to participate in the federation. Considering the a…
FairnessFederated Learning