Uncovering Gradient Inversion Risks in Practical Language Model Training
The gradient inversion attack has been demonstrated as a significant privacy threat to federated learning (FL), particularly in continuous domains such as vision models. In contrast, it is often considered less effective or highly dependent on impractical training settings when applied to language models, due to the challenges posed by the discrete nature of tokens in text data. As a result, its potential privacy threats remain largely underestimated, despite FL being an emerging training method for language models. In this work, we propose a domain-specific gradient inversion attack named Grab (gradient inversion with hybrid optimization). Grab features two alternating optimization processes to address the challenges caused by practical training settings, including a simultaneous optimization on dropout masks between layers for improved token recovery and a discrete optimization for effective token sequencing. Grab can recover a significant portion (up to 92.9% recovery rate) of the private training data, outperforming the attack strategy of utilizing discrete optimization with an auxiliary model by notable improvements of up to 28.9% recovery rate in benchmark settings and 48.5% recovery rate in practical settings. Grab provides a valuable step forward in understanding this privacy threat in the emerging FL training mode of language models.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
Trajectory-Aware Information Matching for Multi-Step Gradient Inversion in Federated Learning
Federated learning enables distributed information sharing and collaborative model training without exposing raw client data. However, shared gradients or model updates may still contain sensitive information, making fed…
Federated LearningTransferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queries
This study investigates the privacy risks associated with text embeddings, focusing on the scenario where attackers cannot access the original embedding model. Contrary to previous research requiring direct model access,…
GI-SMN: Gradient Inversion Attack against Federated Learning without Prior Knowledge
Federated learning (FL) has emerged as a privacy-preserving machine learning approach where multiple parties share gradient information rather than original user data. Recent work has demonstrated that gradient inversion…
Federated LearningPrivacy PreservingDo Vision-Language Models Leak What They Learn? Adaptive Token-Weighted Model Inversion Attacks
Model inversion (MI) attacks pose significant privacy risks by reconstructing private training data from trained neural networks. While prior studies have primarily examined unimodal deep networks, the vulnerability of v…
Image ReconstructionVisual GroundingGI-PIP: Do We Require Impractical Auxiliary Dataset for Gradient Inversion Attacks?
Deep gradient inversion attacks expose a serious threat to Federated Learning (FL) by accurately recovering private data from shared gradients. However, the state-of-the-art heavily relies on impractical assumptions to a…
Anomaly DetectionFederated Learning