Personalized Federated Learning of Probabilistic Models: A PAC-Bayesian Approach
Federated Learning (FL) aims to infer a shared model from private and decentralized data stored by multiple clients. Personalized FL (PFL) enhances the model's fit for each client by adapting the global model to the clients. A significant level of personalization is required for highly heterogeneous clients but can be challenging to achieve, especially when clients' datasets are small. To address this issue, we introduce the PAC-PFL framework for PFL of probabilistic models. PAC-PFL infers a shared hyper-posterior and treats each client's posterior inference as the personalization step. Unlike previous PFL algorithms, PAC-PFL does not regularize all personalized models towards a single shared model, thereby greatly enhancing its personalization flexibility. By establishing and minimizing a PAC-Bayesian generalization bound on the average true loss of clients, PAC-PFL effectively mitigates overfitting even in data-poor scenarios. Additionally, PAC-PFL provides generalization bounds for new clients joining later. PAC-PFL achieves accurate and well-calibrated predictions, as supported by our experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningGeneralization BoundsPersonalized Federated LearningSimilar Papers 제목 키워드 기반
Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization
Conventional federated learning (FL) frameworks often suffer from training degradation due to data uncertainty and heterogeneity across local clients. Probabilistic approaches such as Bayesian neural networks (BNNs) can …
Federated LearningTrustworthy Personalized Bayesian Federated Learning via Posterior Fine-Tune
Performance degradation owing to data heterogeneity and low output interpretability are the most significant challenges faced by federated learning in practical applications. Personalized federated learning diverges from…
Federated LearningOut of Distribution (OOD) DetectionPersonalized Federated LearningBayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representations
Bayesian personalized federated learning (BPFL) addresses challenges in existing personalized FL (PFL). BPFL aims to quantify the uncertainty and heterogeneity within and across clients towards uncertainty representation…
Federated LearningPersonalized Federated LearningBayesian Neural Network For Personalized Federated Learning Parameter Selection
Federated learning's poor performance in the presence of heterogeneous data remains one of the most pressing issues in the field. Personalized federated learning departs from the conventional paradigm in which all client…
Federated LearningPersonalized Federated LearningPersonalized Federated Learning via Variational Bayesian Inference
Federated learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, this paper proposes a novel personalized federated learning me…
Bayesian InferenceDiversityFederated LearningPersonalized Federated Learning+1