SRATTA : Sample Re-ATTribution Attack of Secure Aggregation in Federated Learning
We consider a cross-silo federated learning (FL) setting where a machine learning model with a fully connected first layer is trained between different clients and a central server using FedAvg, and where the aggregation step can be performed with secure aggregation (SA). We present SRATTA an attack relying only on aggregated models which, under realistic assumptions, (i) recovers data samples from the different clients, and (ii) groups data samples coming from the same client together. While sample recovery has already been explored in an FL setting, the ability to group samples per client, despite the use of SA, is novel. This poses a significant unforeseen security threat to FL and effectively breaks SA. We show that SRATTA is both theoretically grounded and can be used in practice on realistic models and datasets. We also propose counter-measures, and claim that clients should play an active role to guarantee their privacy during training.
Code (1)
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
Eluding Secure Aggregation in Federated Learning via Model Inconsistency
Secure aggregation is a cryptographic protocol that securely computes the aggregation of its inputs. It is pivotal in keeping model updates private in federated learning. Indeed, the use of secure aggregation prevents th…
Federated LearningGradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix
We show that aggregated model updates in federated learning may be insecure. An untrusted central server may disaggregate user updates from sums of updates across participants given repeated observations, enabling the se…
Federated LearningLOKI: Large-scale Data Reconstruction Attack against Federated Learning through Model Manipulation
Federated learning was introduced to enable machine learning over large decentralized datasets while promising privacy by eliminating the need for data sharing. Despite this, prior work has shown that shared gradients of…
Federated LearningReconstruction AttackUncovering Attacks and Defenses in Secure Aggregation for Federated Deep Learning
Federated learning enables the collaborative learning of a global model on diverse data, preserving data locality and eliminating the need to transfer user data to a central server. However, data privacy remains vulnerab…
Federated LearningCachePrune: Neural-Based Attribution Defense Against Indirect Prompt Injection Attacks
Large Language Models (LLMs) are identified as being susceptible to indirect prompt injection attack, where the model undesirably deviates from user-provided instructions by executing tasks injected in the prompt context…
Instruction Following