Federated Learning on Adaptively Weighted Nodes by Bilevel Optimization
We propose a federated learning method with weighted nodes in which the weights can be modified to optimize the model's performance on a separate validation set. The problem is formulated as a bilevel optimization where the inner problem is a federated learning problem with weighted nodes and the outer problem focuses on optimizing the weights based on the validation performance of the model returned from the inner problem. A communication-efficient federated optimization algorithm is designed to solve this bilevel optimization problem. Under an error-bound assumption, we analyze the generalization performance of the output model and identify scenarios when our method is in theory superior to training a model only locally and to federated learning with static and evenly distributed weights.
Code (0)
등록된 구현이 없습니다.
Tasks
Bilevel OptimizationFederated LearningSimilar Papers 제목 키워드 기반
Communication-Efficient Federated Bilevel Optimization with Local and Global Lower Level Problems
Bilevel Optimization has witnessed notable progress recently with new emerging efficient algorithms. However, its application in the Federated Learning setting remains relatively underexplored, and the impact of Federate…
Bilevel OptimizationFederated LearningRepresentation LearningCommunication-Efficient Federated Bilevel Optimization with Global and Local Lower Level Problems
Bilevel Optimization has witnessed notable progress recently with new emerging efficient algorithms. However, its application in the Federated Learning setting remains relatively underexplored, and the impact of Federate…
Achieving Linear Speedup in Non-IID Federated Bilevel Learning
Federated bilevel optimization has received increasing attention in various emerging machine learning and communication applications. Recently, several Hessian-vector-based algorithms have been proposed to solve the fede…
Bilevel OptimizationFederated LearningFast Adaptive Federated Bilevel Optimization
Bilevel optimization is a popular hierarchical model in machine learning, and has been widely applied to many machine learning tasks such as meta learning, hyperparameter learning and policy optimization. Although many b…
Bilevel OptimizationDistributed OptimizationMeta-LearningRepresentation LearningOn the Convergence of Momentum-Based Algorithms for Federated Bilevel Optimization Problems
In this paper, we studied the federated bilevel optimization problem, which has widespread applications in machine learning. In particular, we developed two momentum-based algorithms for optimizing this kind of problem a…
Bilevel Optimization