Personalized Federated Learning under Model Dissimilarity Constraints
One of the defining challenges in federated learning is that of statistical heterogeneity among clients. We address this problem with KARULA, a regularized strategy for personalized federated learning, which constrains the pairwise model dissimilarities between clients based on the difference in their distributions, as measured by a surrogate for the 1-Wasserstein distance adapted for the federated setting. This allows the strategy to adapt to highly complex interrelations between clients, that e.g., clustered approaches fail to capture. We propose an inexact projected stochastic gradient algorithm to solve the constrained problem that the strategy defines, and show theoretically that it converges with smooth, possibly non-convex losses to a neighborhood of a stationary point with rate O(1/K). We demonstrate the effectiveness of KARULA on synthetic and real federated data sets.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningmodelPersonalized Federated LearningSimilar Papers 제목 키워드 기반
A Randomized Zeroth-Order Hierarchical Framework for Heterogeneous Federated Learning
Heterogeneity in federated learning (FL) is a critical and challenging aspect that significantly impacts model performance and convergence. In this paper, we propose a novel framework by formulating heterogeneous FL as a…
Federated Learningimage-classificationImage ClassificationAgnostic Personalized Federated Learning with Kernel Factorization
Considering the futuristic scenarios of federated learning at a worldwide scale, it is highly probable that local participants can have their own personalized labels, which might not be compatible with each other even fo…
Federated LearningPersonalized Federated LearningSemantic SimilaritySemantic Textual SimilarityFLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning
Federated Learning (FL) is an increasingly popular machine learning paradigm in which multiple nodes try to collaboratively learn under privacy, communication and multiple heterogeneity constraints. A persistent problem …
Distributed OptimizationFederated LearningPersonalized Federated Learning via Heterogeneous Modular Networks
Personalized Federated Learning (PFL) which collaboratively trains a federated model while considering local clients under privacy constraints has attracted much attention. Despite its popularity, it has been observed th…
Federated LearningPersonalized Federated LearningOn Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond
The FedProx algorithm is a simple yet powerful distributed proximal point optimization method widely used for federated learning (FL) over heterogeneous data. Despite its popularity and remarkable success witnessed in pr…
Federated Learning