Human-Readable Adversarial Prompts: An Investigation into LLM Vulnerabilities Using Situational Context

2024-12-20 · Nilanjana Das, Edward Raff, Manas Gaur

Previous research on LLM vulnerabilities often relied on nonsensical adversarial prompts, which were easily detectable by automated methods. We address this gap by focusing on human-readable adversarial prompts, a more realistic and potent threat. Our key contributions are situation-driven attacks leveraging movie scripts to create contextually relevant, human-readable prompts that successfully deceive LLMs, adversarial suffix conversion to transform nonsensical adversarial suffixes into meaningful text, and AdvPrompter with p-nucleus sampling, a method to generate diverse, human-readable adversarial suffixes, improving attack efficacy in models like GPT-3.5 and Gemma 7B. Our findings demonstrate that LLMs can be tricked by sophisticated adversaries into producing harmful responses with human-readable adversarial prompts and that there exists a scope for improvement when it comes to robust LLMs.

📄 PDF Abstract BibTeX arXiv:2412.16359

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

15 Ways to Contact How can i speak to someone at Delta Airlines 설명 없음

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Weight Decay 설명 없음

Linear Warmup With Cosine Annealing Linear Warmup With Cosine Annealing is a learning rate schedule where we increase the learning rate linearly for $n$ updates and then anneal according to a cosine schedule…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Prompts

2025-10-28 · Yufan Liu, Wanqian Zhang, Huashan Chen, Lin Wang 외 arxiv

Despite rapid advancements in text-to-image (T2I) models, their safety mechanisms are vulnerable to adversarial prompts, which maliciously generate unsafe images. Current red-teaming methods for proactively assessing suc…

AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models

2023-10-23 · Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu 외

Safety alignment of Large Language Models (LLMs) can be compromised with manual jailbreak attacks and (automatic) adversarial attacks. Recent studies suggest that defending against these attacks is possible: adversarial …

Adversarial Attack Blocking Safety Alignment

MMLU Red Teaming

Human-Readable Adversarial Prompts: An Investigation into LLM Vulnerabilities Using Situational Context

Code (0)

Methods 이 논문이 사용한 방법론

Similar Papers 제목 키워드 기반

AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Prompts

AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models

Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation

BayesPrompt: human readable prompts that make sense

AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs