Distilling Adversarial Prompts from Safety Benchmarks: Report for the Adversarial Nibbler Challenge

2023-09-20 · Manuel Brack, Patrick Schramowski, Kristian Kersting

Text-conditioned image generation models have recently achieved astonishing image quality and alignment results. Consequently, they are employed in a fast-growing number of applications. Since they are highly data-driven, relying on billion-sized datasets randomly scraped from the web, they also produce unsafe content. As a contribution to the Adversarial Nibbler challenge, we distill a large set of over 1,000 potential adversarial inputs from existing safety benchmarks. Our analysis of the gathered prompts and corresponding images demonstrates the fragility of input filters and provides further insights into systematic safety issues in current generative image models.

📄 PDF Abstract BibTeX arXiv:2309.11575

Code (0)

등록된 구현이 없습니다.

Tasks

Image Generation

A Safety Report on GPT-5.2, Gemini 3 Pro, Qwen3-VL, Grok 4.1 Fast, Nano Banana Pro, and Seedream 4.5

2026-01-15 · Xingjun Ma, Yixu Wang, Hengyuan Xu, Yutao Wu 외 arxiv

The rapid evolution of Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) has driven major gains in reasoning, perception, and generation across language and vision, yet whether these advances tran…

Adversarial Robustness Image Generation

Distilling Safe LLM Systems via Soft Prompts for On Device Settings

2026-06-08 · Motasem Alfarra, Cristina Pinneri, Dana Kianfar, Mohammed Almousa 외 arxiv

Deploying safe large language models (LLMs) on resource-constrained edge devices presents a critical challenge: while dual-model systems combining LLMs with guard models provide effective safety guarantees, their substan…

parameter-efficient fine-tuning

MART: Improving LLM Safety with Multi-round Automatic Red-Teaming

2023-11-13 · Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa 외

Red-teaming is a common practice for mitigating unsafe behaviors in Large Language Models (LLMs), which involves thoroughly assessing LLMs to identify potential flaws and addressing them with responsible and accurate res…

Instruction Following Red Teaming Response Generation Safety Alignment

CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs

2025-05-16 · Sijia Chen, Xiaomin Li, Mengxue Zhang, Eric Hanchen Jiang 외

Large language models (LLMs) are increasingly deployed in medical contexts, raising critical concerns about safety, alignment, and susceptibility to adversarial manipulation. While prior benchmarks assess model refusal c…

Adversarial Robustness Safety Alignment Specificity

Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection

2026-05-24 · Lixing Lin, Juli You, Yue Li, Luyun Lin 외 arxiv

Large language model (LLM) safety classifiers such as Llama Guard are effective at detecting overtly harmful prompts but remain vulnerable to adversarial jailbreak attacks that disguise malicious intent through role-play…