FedCBO: Reaching Group Consensus in Clustered Federated Learning through Consensus-based Optimization
Federated learning is an important framework in modern machine learning that seeks to integrate the training of learning models from multiple users, each user having their own local data set, in a way that is sensitive to data privacy and to communication loss constraints. In clustered federated learning, one assumes an additional unknown group structure among users, and the goal is to train models that are useful for each group, rather than simply training a single global model for all users. In this paper, we propose a novel solution to the problem of clustered federated learning that is inspired by ideas in consensus-based optimization (CBO). Our new CBO-type method is based on a system of interacting particles that is oblivious to group memberships. Our model is motivated by rigorous mathematical reasoning, including a mean field analysis describing the large number of particles limit of our particle system, as well as convergence guarantees for the simultaneous global optimization of general non-convex objective functions (corresponding to the loss functions of each cluster of users) in the mean-field regime. Experimental results demonstrate the efficacy of our FedCBO algorithm compared to other state-of-the-art methods and help validate our methodological and theoretical work.
Code (1)
Tasks
Federated Learningglobal-optimizationMathematical ReasoningSimilar Papers 제목 키워드 기반
Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces
Clustered federated learning (FL) has been shown to produce promising results by grouping clients into clusters. This is especially effective in scenarios where separate groups of clients have significant differences in …
Federated LearningDefending Against Diverse Attacks in Federated Learning Through Consensus-Based Bi-Level Optimization
Adversarial attacks pose significant challenges in many machine learning applications, particularly in the setting of distributed training and federated learning, where malicious agents seek to corrupt the training proce…
Federated LearningMetaheuristic OptimizationGroup Consensus of Linear Multi-agent Systems under Nonnegative Directed Graphs
Group consensus implies reaching multiple groups where agents belonging to the same cluster reach state consensus. This paper focuses on linear multi-agent systems under nonnegative directed graphs. A new necessary and s…
ClusteringStochastic Clustered Federated Learning
Federated learning is a distributed learning framework that takes full advantage of private data samples kept on edge devices. In real-world federated learning systems, these data samples are often decentralized and Non-…
Federated LearningFedSoft: Soft Clustered Federated Learning with Proximal Local Updating
Traditionally, clustered federated learning groups clients with the same data distribution into a cluster, so that every client is uniquely associated with one data distribution and helps train a model for this distribut…
Federated Learning