Multi-Model Federated Learning
Federated learning is a form of distributed learning with the key challenge being the non-identically distributed nature of the data in the participating clients. In this paper, we extend federated learning to the setting where multiple unrelated models are trained simultaneously. Specifically, every client is able to train any one of M models at a time and the server maintains a model for each of the M models which is typically a suitably averaged version of the model computed by the clients. We propose multiple policies for assigning learning tasks to clients over time. In the first policy, we extend the widely studied FedAvg to multi-model learning by allotting models to clients in an i.i.d. stochastic manner. In addition, we propose two new policies for client selection in a multi-model federated setting which make decisions based on current local losses for each client-model pair. We compare the performance of the policies on tasks involving synthetic and real-world data and characterize the performance of the proposed policies. The key take-away from our work is that the proposed multi-model policies perform better or at least as good as single model training using FedAvg.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningmodelSimilar Papers 제목 키워드 기반
Variational Federated Multi-Task Learning
In federated learning, a central server coordinates the training of a single model on a massively distributed network of devices. This setting can be naturally extended to a multi-task learning framework, to handle real-…
Federated LearningMulti-Task LearningVariational InferenceFLAG: Fast Label-Adaptive Aggregation for Multi-label Classification in Federated Learning
Federated learning aims to share private data to maximize the data utility without privacy leakage. Previous federated learning research mainly focuses on multi-class classification problems. However, multi-label classif…
ClassificationFederated LearningMulti-class ClassificationMulti-Label Classification+1Federated Multi-view Matrix Factorization for Personalized Recommendations
We introduce the federated multi-view matrix factorization method that extends the federated learning framework to matrix factorization with multiple data sources. Our method is able to learn the multi-view model without…
Federated LearningFederated Multi-Mini-Batch: An Efficient Training Approach to Federated Learning in Non-IID Environments
Federated learning has faced performance and network communication challenges, especially in the environments where the data is not independent and identically distributed (IID) across the clients. To address the former …
Federated LearningFederated Multi-Objective Learning
In recent years, multi-objective optimization (MOO) emerges as a foundational problem underpinning many multi-agent multi-task learning applications. However, existing algorithms in MOO literature remain limited to centr…
Federated LearningMulti-Task Learning