WAFFLE: Weighted Averaging for Personalized Federated Learning
In federated learning, model personalization can be a very effective strategy to deal with heterogeneous training data across clients. We introduce WAFFLE (Weighted Averaging For Federated LEarning), a personalized collaborative machine learning algorithm that leverages stochastic control variates for faster convergence. WAFFLE uses the Euclidean distance between clients' updates to weigh their individual contributions and thus minimize the personalized model loss on the specific agent of interest. Through a series of experiments, we compare our new approach to two recent personalized federated learning methods--Weight Erosion and APFL--as well as two general FL methods--Federated Averaging and SCAFFOLD. Performance is evaluated using two categories of non-identical client data distributions--concept shift and label skew--on two image data sets (MNIST and CIFAR10). Our experiments demonstrate the comparative effectiveness of WAFFLE, as it achieves or improves accuracy with faster convergence.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningPersonalized Federated LearningSimilar Papers 제목 키워드 기반
Optimal Model Averaging: Towards Personalized Collaborative Learning
In federated learning, differences in the data or objectives between the participating nodes motivate approaches to train a personalized machine learning model for each node. One such approach is weighted averaging betwe…
Federated Learningmodelparameter estimationWAFFLe: Weight Anonymized Factorization for Federated Learning
In domains where data are sensitive or private, there is great value in methods that can learn in a distributed manner without the data ever leaving the local devices. In light of this need, federated learning has emerge…
FairnessFederated LearningAdaptive Personalized Federated Learning via Multi-task Averaging of Kernel Mean Embeddings
Personalized Federated Learning (PFL) enables a collection of agents to collaboratively learn individual models without sharing raw data. We propose a new PFL approach in which each agent optimizes a weighted combination…
Personalized Federated LearningRegularizing and Aggregating Clients with Class Distribution for Personalized Federated Learning
Personalized federated learning (PFL) enables customized models for clients with varying data distributions. However, existing PFL methods often incur high computational and communication costs, limiting their practical …
Federated LearningPersonalized Federated LearningWaffling around for Performance: Visual Classification with Random Words and Broad Concepts
The visual classification performance of vision-language models such as CLIP has been shown to benefit from additional semantic knowledge from large language models (LLMs) such as GPT-3. In particular, averaging over LLM…
ClassificationLanguage ModelingLanguage ModellingRetrieval