Backdoor Learning Curves: Explaining Backdoor Poisoning Beyond Influence Functions
Backdoor attacks inject poisoning samples during training, with the goal of forcing a machine learning model to output an attacker-chosen class when presented a specific trigger at test time. Although backdoor attacks have been demonstrated in a variety of settings and against different models, the factors affecting their effectiveness are still not well understood. In this work, we provide a unifying framework to study the process of backdoor learning under the lens of incremental learning and influence functions. We show that the effectiveness of backdoor attacks depends on: (i) the complexity of the learning algorithm, controlled by its hyperparameters; (ii) the fraction of backdoor samples injected into the training set; and (iii) the size and visibility of the backdoor trigger. These factors affect how fast a model learns to correlate the presence of the backdoor trigger with the target class. Our analysis unveils the intriguing existence of a region in the hyperparameter space in which the accuracy on clean test samples is still high while backdoor attacks are ineffective, thereby suggesting novel criteria to improve existing defenses.
Code (1)
Tasks
BIG-bench Machine LearningIncremental LearningSimilar Papers 제목 키워드 기반
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Deep learning models have achieved high performance on many tasks, and thus have been applied to many security-critical scenarios. For example, deep learning-based face recognition systems have been used to authenticate …
Data PoisoningDeep LearningFace RecognitionBoosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategy
Data-poisoning based backdoor attacks aim to insert backdoor into models by manipulating training datasets without controlling the training process of the target model. Existing attack methods mainly focus on designing t…
Backdoor AttackData PoisoningDefending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning
Recently, various parameter-efficient fine-tuning (PEFT) strategies for application to language models have been proposed and successfully implemented. However, this raises the question of whether PEFT, which only update…
Backdoor Attackparameter-efficient fine-tuningtext-classificationText ClassificationMitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification
It has been proved that deep neural networks are facing a new threat called backdoor attacks, where the adversary can inject backdoors into the neural network model through poisoning the training dataset. When the input …
ClassificationData PoisoningGeneral Classificationtext-classification+1When Backdoors Go Beyond Triggers: Semantic Drift in Diffusion Models Under Encoder Attacks
Standard evaluations of backdoor attacks on text-to-image (T2I) models primarily measure trigger activation and visual fidelity. We challenge this paradigm, demonstrating that encoder-side poisoning induces persistent, t…