A Survey of Recent Backdoor Attacks and Defenses in Large Language Models
Large Language Models (LLMs), which bridge the gap between human language understanding and complex problem-solving, achieve state-of-the-art performance on several NLP tasks, particularly in few-shot and zero-shot settings. Despite the demonstrable efficacy of LLMs, due to constraints on computational resources, users have to engage with open-source language models or outsource the entire training process to third-party platforms. However, research has demonstrated that language models are susceptible to potential security vulnerabilities, particularly in backdoor attacks. Backdoor attacks are designed to introduce targeted vulnerabilities into language models by poisoning training samples or model weights, allowing attackers to manipulate model responses through malicious triggers. While existing surveys on backdoor attacks provide a comprehensive overview, they lack an in-depth examination of backdoor attacks specifically targeting LLMs. To bridge this gap and grasp the latest trends in the field, this paper presents a novel perspective on backdoor attacks for LLMs by focusing on fine-tuning methods. Specifically, we systematically classify backdoor attacks into three categories: full-parameter fine-tuning, parameter-efficient fine-tuning, and no fine-tuning Based on insights from a substantial review, we also discuss crucial issues for future research on backdoor attacks, such as further exploring attack algorithms that do not require fine-tuning, or developing more covert attack algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
parameter-efficient fine-tuningSimilar Papers 제목 키워드 기반
Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions
Federated learning (FL) is a machine learning (ML) approach that allows the use of distributed data without compromising personal privacy. However, the heterogeneous distribution of data among clients in FL can make it d…
Backdoor AttackFederated LearningSurveyA Survey on Backdoor Threats in Large Language Models (LLMs): Attacks, Defenses, and Evaluations
Large Language Models (LLMs) have achieved significantly advanced capabilities in understanding and generating human language text, which have gained increasing popularity over recent years. Apart from their state-of-the…
Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs
With the rise of advanced reasoning capabilities, large language models (LLMs) are receiving increasing attention. However, although reasoning improves LLMs' performance on downstream tasks, it also introduces new securi…
On Evaluating Neural Network Backdoor Defenses
Deep neural networks (DNNs) demonstrate superior performance in various fields, including scrutiny and security. However, recent studies have shown that DNNs are vulnerable to backdoor attacks. Several defenses were prop…
Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff
Data poisoning and backdoor attacks manipulate victim models by maliciously modifying training data. In light of this growing threat, a recent survey of industry professionals revealed heightened fear in the private sect…
Data AugmentationData Poisoning