Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generator

2024-09-11 · Kangyang Luo, Shuai Wang, Xiang Li, Yunshi Lan, Ming Gao, Jinlong Shu

Federated Learning (FL) is gaining popularity as a distributed learning framework that only shares model parameters or gradient updates and keeps private data locally. However, FL is at risk of privacy leakage caused by privacy inference attacks. And most existing privacy-preserving mechanisms in FL conflict with achieving high performance and efficiency. Therefore, we propose FedMD-CG, a novel FL method with highly competitive performance and high-level privacy preservation, which decouples each client's local model into a feature extractor and a classifier, and utilizes a conditional generator instead of the feature extractor to perform server-side model aggregation. To ensure the consistency of local generators and classifiers, FedMD-CG leverages knowledge distillation to train local models and generators at both the latent feature level and the logit level. Also, we construct additional classification losses and design new diversity losses to enhance client-side training. FedMD-CG is robust to data heterogeneity and does not require training extra discriminators (like cGAN). We conduct extensive experiments on various image classification tasks to validate the superiority of FedMD-CG.

📄 PDF Abstract BibTeX arXiv:2409.06955

Code (0)

등록된 구현이 없습니다.

Tasks

DiversityFederated Learning image-classification Image Classification Knowledge DistillationPrivacy Preserving

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation

2022-09-10 · Xuan Gong, Abhishek Sharma, Srikrishna Karanam, Ziyan Wu 외

Federated Learning (FL) is a machine learning paradigm where local nodes collaboratively train a central model while the training data remains decentralized. Existing FL methods typically share model parameters or employ…

Federated Learning image-classification Image Classification Knowledge Distillation+3

Selective Knowledge Sharing for Privacy-Preserving Federated Distillation without A Good Teacher

2023-04-04 · Jiawei Shao, Fangzhao Wu, Jun Zhang

While federated learning is promising for privacy-preserving collaborative learning without revealing local data, it remains vulnerable to white-box attacks and struggles to adapt to heterogeneous clients. Federated dist…

Federated Learning Knowledge Distillation Privacy Preserving Transfer Learning

Federated Learning with Privacy-Preserving Ensemble Attention Distillation

2022-10-16 · Xuan Gong, Liangchen Song, Rishi Vedula, Abhishek Sharma 외

Federated Learning (FL) is a machine learning paradigm where many local nodes collaboratively train a central model while keeping the training data decentralized. This is particularly relevant for clinical applications s…

Federated Learning image-classification Image Classification Knowledge Distillation+1

Better Knowledge Enhancement for Privacy-Preserving Cross-Project Defect Prediction

2024-12-23 · Yuying Wang, Yichen Li, Haozhao Wang, Lei Zhao 외

Cross-Project Defect Prediction (CPDP) poses a non-trivial challenge to construct a reliable defect predictor by leveraging data from other projects, particularly when data owners are concerned about data privacy. In rec…

Federated Learning Knowledge Distillation Privacy Preserving

FedPDD: A Privacy-preserving Double Distillation Framework for Cross-silo Federated Recommendation

2023-05-09 · Sheng Wan, Dashan Gao, Hanlin Gu, Daning Hu

Cross-platform recommendation aims to improve recommendation accuracy by gathering heterogeneous features from different platforms. However, such cross-silo collaborations between platforms are restricted by increasingly…

Federated Learning Privacy Preserving