Open Sesame! Universal Black Box Jailbreaking of Large Language Models

2023-09-04 · Raz Lapid, Ron Langberg, Moshe Sipper

Large language models (LLMs), designed to provide helpful and safe responses, often rely on alignment techniques to align with user intent and social guidelines. Unfortunately, this alignment can be exploited by malicious actors seeking to manipulate an LLM's outputs for unintended purposes. In this paper we introduce a novel approach that employs a genetic algorithm (GA) to manipulate LLMs when model architecture and parameters are inaccessible. The GA attack works by optimizing a universal adversarial prompt that -- when combined with a user's query -- disrupts the attacked model's alignment, resulting in unintended and potentially harmful outputs. Our novel approach systematically reveals a model's limitations and vulnerabilities by uncovering instances where its responses deviate from expected behavior. Through extensive experiments we demonstrate the efficacy of our technique, thus contributing to the ongoing discussion on responsible AI development by providing a diagnostic tool for evaluating and enhancing alignment of LLMs with human intent. To our knowledge this is the first automated universal black box jailbreak attack.

📄 PDF Abstract BibTeX arXiv:2309.01446

Code (0)

등록된 구현이 없습니다.

Tasks

Diagnostic

Methods 이 논문이 사용한 방법론

ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

GA Genetic Algorithms are search algorithms that mimic Darwinian biological evolution in order to select and propagate better solutions.

Jailbreaking to Jailbreak

2025-02-09 · Jeremy Kritz, Vaughn Robinson, Robert Vacareanu, Bijan Varjavand 외

Large Language Models (LLMs) can be used to red team other models (e.g. jailbreaking) to elicit harmful contents. While prior works commonly employ open-weight models or private uncensored models for doing jailbreaking, …

Jailbreaking Attack against Multimodal Large Language Model

2024-02-04 · Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua 외

This paper focuses on jailbreaking attacks against multi-modal large language models (MLLMs), seeking to elicit MLLMs to generate objectionable responses to harmful user queries. A maximum likelihood-based algorithm is p…

Language Modeling Language Modelling Large Language Model model+1

AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization

2024-05-30 · Jiawei Chen, Xiao Yang, Zhengwei Fang, Yu Tian 외

Despite the widespread application of large language models (LLMs) across various tasks, recent studies indicate that they are susceptible to jailbreak attacks, which can render their defense mechanisms ineffective. Howe…

Sentence Sentence Compression

Breaking Bad: Interpretability-Based Safety Audits of State-of-the-Art LLMs

2026-04-22 · Krishiv Agarwal, Ramneet Kaur, Colin Samplawski, Manoj Acharya 외 arxiv

Effective safety auditing of large language models (LLMs) demands tools that go beyond black-box probing and systematically uncover vulnerabilities rooted in model internals. We present a comprehensive, interpretability-…

Sesame Plant Segmentation Dataset: A YOLO Formatted Annotated Dataset

2026-01-12 · Sunusi Ibrahim Muhammad, Ismail Ismail Tijjani, Saadatu Yusuf Jumare, Fatima Isah Jibrin arxiv

This paper presents the Sesame Plant Segmentation Dataset, an open source annotated image dataset designed to support the development of artificial intelligence models for agricultural applications, with a specific focus…