PGFed: Personalize Each Client's Global Objective for Federated Learning
Personalized federated learning has received an upsurge of attention due to the mediocre performance of conventional federated learning (FL) over heterogeneous data. Unlike conventional FL which trains a single global consensus model, personalized FL allows different models for different clients. However, existing personalized FL algorithms only implicitly transfer the collaborative knowledge across the federation by embedding the knowledge into the aggregated model or regularization. We observed that this implicit knowledge transfer fails to maximize the potential of each client's empirical risk toward other clients. Based on our observation, in this work, we propose Personalized Global Federated Learning (PGFed), a novel personalized FL framework that enables each client to personalize its own global objective by explicitly and adaptively aggregating the empirical risks of itself and other clients. To avoid massive (O(N^2)) communication overhead and potential privacy leakage while achieving this, each client's risk is estimated through a first-order approximation for other clients' adaptive risk aggregation. On top of PGFed, we develop a momentum upgrade, dubbed PGFedMo, to more efficiently utilize clients' empirical risks. Our extensive experiments on four datasets under different federated settings show consistent improvements of PGFed over previous state-of-the-art methods. The code is publicly available at https://github.com/ljaiverson/pgfed.
Code (1)
Tasks
Federated LearningPersonalized Federated LearningTransfer LearningSimilar Papers 제목 키워드 기반
Separate Aggregation of Split Network for Personalized Federated Learning
Federated learning enables collaborative model training without sharing raw data, but its performance can degrade substantially under heterogeneous client data distributions. A single global model often cannot satisfy di…
Personalized Federated LearningPersonalized Federated Knowledge Graph Embedding with Client-Wise Relation Graph
Federated Knowledge Graph Embedding (FKGE) has recently garnered considerable interest due to its capacity to extract expressive representations from distributed knowledge graphs, while concurrently safeguarding the priv…
Entity Embeddingsglobal-optimizationGraph EmbeddingKnowledge Graph Embedding+2Personalized Federated Learning with Moreau Envelopes
Federated learning (FL) is a decentralized and privacy-preserving machine learning technique in which a group of clients collaborate with a server to learn a global model without sharing clients' data. One challenge asso…
DiversityFederated LearningMeta-LearningModel Optimization+3ActPerFL: Active Personalized Federated Learning
In the context of personalized federated learning (FL), the critical challenge is to balance local model improvement and global model tuning when the personal and global objectives may not be exactly aligned. Inspired by…
Federated LearningPersonalized Federated LearningUncertainty QuantificationPersonalized Federated Learning via Gaussian Generative Modeling
Federated learning has emerged as a paradigm to train models collaboratively on inherently distributed client data while safeguarding privacy. In this context, personalized federated learning tackles the challenge of dat…
Personalized Federated LearningRepresentation LearningBayesian Inference