Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation
Federated Learning (FL) is a machine learning paradigm where local nodes collaboratively train a central model while the training data remains decentralized. Existing FL methods typically share model parameters or employ co-distillation to address the issue of unbalanced data distribution. However, they suffer from communication bottlenecks. More importantly, they risk privacy leakage. In this work, we develop a privacy preserving and communication efficient method in a FL framework with one-shot offline knowledge distillation using unlabeled, cross-domain public data. We propose a quantized and noisy ensemble of local predictions from completely trained local models for stronger privacy guarantees without sacrificing accuracy. Based on extensive experiments on image classification and text classification tasks, we show that our privacy-preserving method outperforms baseline FL algorithms with superior performance in both accuracy and communication efficiency.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated Learningimage-classificationImage ClassificationKnowledge DistillationPrivacy Preservingtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FedED: Federated Learning via Ensemble Distillation for Medical Relation Extraction
Unlike other domains, medical texts are inevitably accompanied by private information, so sharing or copying these texts is strictly restricted. However, training a medical relation extraction model requires collecting t…
Federated LearningKnowledge DistillationMedical Relation ExtractionPrivacy Preserving+2Federated Cross-Domain Click-Through Rate Prediction With Large Language Model Augmentation
Accurately predicting click-through rates (CTR) under stringent privacy constraints poses profound challenges, particularly when user-item interactions are sparse and fragmented across domains. Conventional cross-domain …
Click-Through Rate PredictionContrastive LearningData AugmentationDisentanglement+5Quantum Privacy Aggregation of Teacher Ensembles (QPATE) for Privacy-preserving Quantum Machine Learning
The utility of machine learning has rapidly expanded in the last two decades and presents an ethical challenge. Papernot et. al. developed a technique, known as Private Aggregation of Teacher Ensembles (PATE) to enable f…
Federated LearningPrivacy PreservingQuantum Machine LearningMulti-Modal One-Shot Federated Ensemble Learning for Medical Data with Vision Large Language Model
Federated learning (FL) has attracted considerable interest in the medical domain due to its capacity to facilitate collaborative model training while maintaining data privacy. However, conventional FL methods typically …
DiagnosticEnsemble LearningFederated LearningLanguage Modeling+3Federated Inference: Toward Privacy-Preserving Collaborative and Incentivized Model Serving
Federated Inference (FI) studies how independently trained and privately owned models can collaborate at inference time without sharing data or model parameters. While recent work has explored secure and distributed infe…
Federated Learning