DROP: Poison Dilution via Knowledge Distillation for Federated Learning

2025-02-10 · Georgios Syros, Anshuman Suri, Farinaz Koushanfar, Cristina Nita-Rotaru, Alina Oprea

Federated Learning is vulnerable to adversarial manipulation, where malicious clients can inject poisoned updates to influence the global model's behavior. While existing defense mechanisms have made notable progress, they fail to protect against adversaries that aim to induce targeted backdoors under different learning and attack configurations. To address this limitation, we introduce DROP (Distillation-based Reduction Of Poisoning), a novel defense mechanism that combines clustering and activity-tracking techniques with extraction of benign behavior from clients via knowledge distillation to tackle stealthy adversaries that manipulate low data poisoning rates and diverse malicious client ratios within the federation. Through extensive experimentation, our approach demonstrates superior robustness compared to existing defenses across a wide range of learning configurations. Finally, we evaluate existing defenses and our method under the challenging setting of non-IID client data distribution and highlight the challenges of designing a resilient FL defense in this setting.

📄 PDF Abstract BibTeX arXiv:2502.07011

Code (1)

gsiros/drop 공식 구현 pytorch

Tasks

Data PoisoningFederated Learning Knowledge Distillation

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Peak-Controlled Logits Poisoning Attack in Federated Distillation

2024-07-25 · Yuhan Tang, Aoxu Zhang, Zhiyuan Wu, Bo Gao 외

Federated Distillation (FD) offers an innovative approach to distributed machine learning, leveraging knowledge distillation for efficient and flexible cross-device knowledge transfer without necessitating the upload of …

Knowledge Distillation Transfer Learning

Logits Poisoning Attack in Federated Distillation

2024-01-08 · Yuhan Tang, Zhiyuan Wu, Bo Gao, Tian Wen 외

Federated Distillation (FD) is a novel and promising distributed machine learning paradigm, where knowledge distillation is leveraged to facilitate a more efficient and flexible cross-device knowledge transfer in federat…

Federated Learning Knowledge Distillation Transfer Learning

Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integration

2024-09-27 · Mahdi Morafah, Vyacheslav Kungurtsev, Hojin Chang, Chen Chen 외

Federated Learning has emerged as a promising paradigm for collaborative machine learning, while preserving user data privacy. Despite its potential, standard FL lacks support for diverse heterogeneous device prototypes,…

Federated Learning Knowledge Distillation Task Arithmetic Transfer Learning

FedD2S: Personalized Data-Free Federated Knowledge Distillation

2024-02-16 · Kawa Atapour, S. Jamal Seyedmohammadi, Jamshid Abouei, Arash Mohammadi 외

This paper addresses the challenge of mitigating data heterogeneity among clients within a Federated Learning (FL) framework. The model-drift issue, arising from the noniid nature of client data, often results in subopti…

Data-free Knowledge Distillation Fairness Federated Learning Knowledge Distillation+1

SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillation

2025-07-11 · Haotian Xu, Jinrui Zhou, Xichong Zhang, Mingjun Xiao 외

Federated Learning (FL) is a distributed machine learning paradigm which coordinates multiple clients to collaboratively train a global model via a central server. Sequential Federated Learning (SFL) is a newly-emerging …

Federated Learning Knowledge Distillation