Privacy Backdoors: Stealing Data with Corrupted Pretrained Models
Practitioners commonly download pretrained machine learning models from open repositories and finetune them to fit specific applications. We show that this practice introduces a new risk of privacy backdoors. By tampering with a pretrained model's weights, an attacker can fully compromise the privacy of the finetuning data. We show how to build privacy backdoors for a variety of models, including transformers, which enable an attacker to reconstruct individual finetuning samples, with a guaranteed success! We further show that backdoored models allow for tight privacy attacks on models trained with differential privacy (DP). The common optimistic practice of training DP models with loose privacy guarantees is thus insecure if the model is not trusted. Overall, our work highlights a crucial and overlooked supply chain attack on machine learning privacy.
Code (1)
Similar Papers 제목 키워드 기반
SAB:A Stealing and Robust Backdoor Attack based on Steganographic Algorithm against Federated Learning
Federated learning, an innovative network architecture designed to safeguard user privacy, is gaining widespread adoption in the realm of technology. However, given the existence of backdoor attacks in federated learning…
Backdoor Attackbackdoor defenseFederated LearningSecret Stealing Attacks on Local LLM Fine-Tuning through Supply-Chain Model Code Backdoors
Local fine-tuning datasets routinely contain sensitive secrets such as API keys, personal identifiers, and financial records. Although "local offline fine-tuning" is often viewed as a privacy boundary, we reveal that com…
Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation
For nearly a decade the academic community has investigated backdoors in neural networks, primarily focusing on classification tasks where adversaries manipulate the model prediction. While demonstrably malicious, the im…
Model Stealing Attack against Recommender System
Recent studies have demonstrated the vulnerability of recommender systems to data privacy attacks. However, research on the threat to model privacy in recommender systems, such as model stealing attacks, is still in its …
modelRecommendation SystemsLMSanitator: Defending Prompt-Tuning Against Task-Agnostic Backdoors
Prompt-tuning has emerged as an attractive paradigm for deploying large-scale language models due to its strong downstream task performance and efficient multitask serving ability. Despite its wide adoption, we empirical…