Adaptive Personalized Federated Learning via Multi-task Averaging of Kernel Mean Embeddings
Personalized Federated Learning (PFL) enables a collection of agents to collaboratively learn individual models without sharing raw data. We propose a new PFL approach in which each agent optimizes a weighted combination of all agents' empirical risks, with the weights learned from data rather than specified a priori. The novelty of our method lies in formulating the estimation of these collaborative weights as a kernel mean embedding estimation problem with multiple data sources, leveraging tools from multi-task averaging to capture statistical relationships between agents. This perspective yields a fully adaptive procedure that requires no prior knowledge of data heterogeneity and can automatically transition between global and local learning regimes. By recasting the objective as a high-dimensional mean estimation problem, we derive finite-sample guarantees on local excess risks for a broad class of distributions, explicitly quantifying the statistical gains of collaboration. To address communication constraints inherent to federated settings, we also propose a practical implementation based on random Fourier features, which allows one to trade communication cost for statistical efficiency. Numerical experiments validate our theoretical results.
Code (0)
등록된 구현이 없습니다.
Tasks
Personalized Federated LearningSimilar Papers 제목 키워드 기반
WAFFLE: Weighted Averaging for Personalized Federated Learning
In federated learning, model personalization can be a very effective strategy to deal with heterogeneous training data across clients. We introduce WAFFLE (Weighted Averaging For Federated LEarning), a personalized colla…
Federated LearningPersonalized Federated LearningFedBiCross: Personalized One-Shot Federated Learning on Medical Images
Data-free knowledge distillation-based one-shot federated learning (OSFL) trains a model in a single communication round without sharing raw data, making OSFL attractive for privacy-sensitive medical applications. Howeve…
Data-free Knowledge DistillationFederated LearningFedMOA: Federated GRPO for Personalized Reasoning LLMs under Heterogeneous Rewards
Group Relative Policy Optimization (GRPO) has recently emerged as an effective approach for improving the reasoning capabilities of large language models through online multi-objective reinforcement learning. While perso…
Reinforcement LearningMathematical ReasoningFederated LearningCode GenerationOptimal Model Averaging: Towards Personalized Collaborative Learning
In federated learning, differences in the data or objectives between the participating nodes motivate approaches to train a personalized machine learning model for each node. One such approach is weighted averaging betwe…
Federated Learningmodelparameter estimationFedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networks
Personalized federated learning is aimed at allowing numerous clients to train personalized models while participating in collaborative training in a communication-efficient manner without exchanging private data. Howeve…
Federated LearningPersonalized Federated LearningRepresentation LearningTransfer Learning