paper-with-me

홈 › Papers

Differential Privacy and Byzantine Resilience in SGD: Do They Add Up?

2021-02-16 · Rachid Guerraoui, Nirupam Gupta, Rafaël Pinot, Sébastien Rouault, John Stephan

This paper addresses the problem of combining Byzantine resilience with privacy in machine learning (ML). Specifically, we study if a distributed implementation of the renowned Stochastic Gradient Descent (SGD) learning algorithm is feasible with both differential privacy (DP) and $(\alpha,f)$-Byzantine resilience. To the best of our knowledge, this is the first work to tackle this problem from a theoretical point of view. A key finding of our analyses is that the classical approaches to these two (seemingly) orthogonal issues are incompatible. More precisely, we show that a direct composition of these techniques makes the guarantees of the resulting SGD algorithm depend unfavourably upon the number of parameters of the ML model, making the training of large models practically infeasible. We validate our theoretical results through numerical experiments on publicly-available datasets; showing that it is impractical to ensure DP and Byzantine resilience simultaneously.

📄 PDF Abstract BibTeX arXiv:2102.08166

Code (1)

LPD-EPFL/DifferentialByzantine 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

SGD Stochastic Gradient Descent is an iterative optimization technique that uses minibatches of data to form an expectation of the gradient, rather than the full gradient using…

Similar Papers 제목 키워드 기반

Practical Differentially Private and Byzantine-resilient Federated Learning

2023-04-15 · Zihang Xiang, Tianhao Wang, WanYu Lin, Di Wang

Privacy and Byzantine resilience are two indispensable requirements for a federated learning (FL) system. Although there have been extensive studies on privacy and Byzantine security in their own track, solutions that co…

Federated LearningPrivacy Preserving

TernaryVote: Differentially Private, Communication Efficient, and Byzantine Resilient Distributed Optimization on Heterogeneous Data

2024-02-16 · Richeng Jin, Yujie Gu, Kai Yue, Xiaofan He 외

Distributed training of deep neural networks faces three critical challenges: privacy preservation, communication efficiency, and robustness to fault and adversarial behaviors. Although significant research efforts have …

Distributed Optimization

Combining Differential Privacy and Byzantine Resilience in Distributed SGD

2021-10-08 · Rachid Guerraoui, Nirupam Gupta, Rafael Pinot, Sebastien Rouault 외

Privacy and Byzantine resilience (BR) are two crucial requirements of modern-day distributed machine learning. The two concepts have been extensively studied individually but the question of how to combine them effective…

Hyperparameter Optimization

Perfect Privacy for Discriminator-Based Byzantine-Resilient Federated Learning

2025-06-16 · Yue Xia, Christoph Hofmeister, Maximilian Egger, Rawad Bitar

Federated learning (FL) shows great promise in large-scale machine learning but introduces new privacy and security challenges. We propose ByITFL and LoByITFL, two novel FL schemes that enhance resilience against Byzanti…

Federated Learning

Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacy

2023-09-07 · Zikai Zhang, Rui Hu

Federated learning (FL) is designed to preserve data privacy during model training, where the data remains on the client side (i.e., IoT devices), and only model updates of clients are shared iteratively for collaborativ…

Federated Learning