paper-with-me

홈 › Papers

One Trigger Token Is Enough: A Defense Strategy for Balancing Safety and Usability in Large Language Models

2025-05-12 · Haoran Gu, Handing Wang, Yi Mei, Mengjie Zhang, Yaochu Jin

Large Language Models (LLMs) have been extensively used across diverse domains, including virtual assistants, automated code generation, and scientific research. However, they remain vulnerable to jailbreak attacks, which manipulate the models into generating harmful responses despite safety alignment. Recent studies have shown that current safety-aligned LLMs often undergo the shallow safety alignment, where the first few tokens largely determine whether the response will be harmful. Through comprehensive observations, we find that safety-aligned LLMs and various defense strategies generate highly similar initial tokens in their refusal responses, which we define as safety trigger tokens. Building on this insight, we propose \texttt{D-STT}, a simple yet effective defense algorithm that identifies and explicitly decodes safety trigger tokens of the given safety-aligned LLM to trigger the model's learned safety patterns. In this process, the safety trigger is constrained to a single token, which effectively preserves model usability by introducing minimum intervention in the decoding process. Extensive experiments across diverse jailbreak attacks and benign prompts demonstrate that \ours significantly reduces output harmfulness while preserving model usability and incurring negligible response time overhead, outperforming ten baseline methods.

📄 PDF Abstract BibTeX arXiv:2505.07167

Code (0)

등록된 구현이 없습니다.

Tasks

Code GenerationSafety Alignment

Similar Papers 제목 키워드 기반

Defense Against Syntactic Textual Backdoor Attacks with Token Substitution

2024-07-04 · Xinglin Li, Xianwen He, Yao Li, Minhao Cheng

Textual backdoor attacks present a substantial security risk to Large Language Models (LLM). It embeds carefully chosen triggers into a victim model at the training stage, and makes the model erroneously predict inputs c…

backdoor defenseSentence

PEPPER: Perception-Guided Perturbation for Robust Backdoor Defense in Text-to-Image Diffusion Models

2025-11-20 · Oscar Chew, Po-Yi Lu, Jayden Lin, Kuan-Hao Huang 외 arxiv

Recent studies show that text to image (T2I) diffusion models are vulnerable to backdoor attacks, where a trigger in the input prompt can steer generation toward harmful or unintended content. Beyond the trigger token it…

FedBAP: Backdoor Defense via Benign Adversarial Perturbation in Federated Learning

2025-07-26 · Xinhai Yan, Libing Wu, Zhuangzhuang Zhang, Bingyi Liu 외 arxiv

Federated Learning (FL) enables collaborative model training while preserving data privacy, but it is highly vulnerable to backdoor attacks. Most existing defense methods in FL have limited effectiveness due to their neg…

Federated Learning

Backdoor defense, learnability and obfuscation

2024-09-04 · Paul Christiano, Jacob Hilton, Victor Lecomte, Mark Xu

We introduce a formal notion of defendability against backdoors using a game between an attacker and a defender. In this game, the attacker modifies a function to behave differently on a particular input known as the "tr…

backdoor defense

GREAT: Generalizable Backdoor Attacks in RLHF via Emotion-Aware Trigger Synthesis

2025-10-10 · Subrat Kishore Dutta, Yuelin Xu, Piyush Pant, Xiao Zhang arxiv

Recent work has shown that RLHF is highly susceptible to backdoor attacks. However, existing methods often rely on rare tokens or fixed triggers, limiting their impact in realistic scenarios. In this work, we develop GRE…

Dimensionality ReductionResponse Generation