Fine-Tuning Personalization in Federated Learning to Mitigate Adversarial Clients
Federated learning (FL) is an appealing paradigm that allows a group of machines (a.k.a. clients) to learn collectively while keeping their data local. However, due to the heterogeneity between the clients' data distributions, the model obtained through the use of FL algorithms may perform poorly on some client's data. Personalization addresses this issue by enabling each client to have a different model tailored to their own data while simultaneously benefiting from the other clients' data. We consider an FL setting where some clients can be adversarial, and we derive conditions under which full collaboration fails. Specifically, we analyze the generalization performance of an interpolated personalized FL framework in the presence of adversarial clients, and we precisely characterize situations when full collaboration performs strictly worse than fine-tuned personalization. Our analysis determines how much we should scale down the level of collaboration, according to data heterogeneity and the tolerable fraction of adversarial clients. We support our findings with empirical results on mean estimation and binary classification problems, considering synthetic and benchmark image classification datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationFederated Learningimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
Robust and Personalized Federated Learning with Spurious Features: an Adversarial Approach
A common approach for personalized federated learning is fine-tuning the global machine learning model to each local client. While this addresses some issues of statistical heterogeneity, we find that such personalizatio…
Federated LearningPersonalized Federated LearningA Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning
Federated Learning (FL) enables decentralized, privacy-preserving model training but struggles to balance global generalization and local personalization due to non-identical data distributions across clients. Personaliz…
Federated LearningProfit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuning
In many applications of federated learning (FL), clients desire models that are personalized using their local data, yet are also robust in the sense that they retain general global knowledge. However, the presence of da…
BenchmarkingFederated LearningGeneral KnowledgeNavigate+1Bi-level Personalization for Federated Foundation Models: A Task-vector Aggregation Approach
Federated foundation models represent a new paradigm to jointly fine-tune pre-trained foundation models across clients. It is still a challenge to fine-tune foundation models for a small group of new users or specialized…
Federated Marginal Personalization for ASR Rescoring
We introduce federated marginal personalization (FMP), a novel method for continuously updating personalized neural network language models (NNLMs) on private devices using federated learning (FL). Instead of fine-tuning…
Federated Learningspeech-recognitionSpeech Recognition