What Really Matters in Many-Shot Attacks? An Empirical Study of Long-Context Vulnerabilities in LLMs
We investigate long-context vulnerabilities in Large Language Models (LLMs) through Many-Shot Jailbreaking (MSJ). Our experiments utilize context length of up to 128K tokens. Through comprehensive analysis with various many-shot attack settings with different instruction styles, shot density, topic, and format, we reveal that context length is the primary factor determining attack effectiveness. Critically, we find that successful attacks do not require carefully crafted harmful content. Even repetitive shots or random dummy text can circumvent model safety measures, suggesting fundamental limitations in long-context processing capabilities of LLMs. The safety behavior of well-aligned models becomes increasingly inconsistent with longer contexts. These findings highlight significant safety gaps in context expansion capabilities of LLMs, emphasizing the need for new safety mechanisms.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?
Building 3D perception systems for autonomous vehicles that do not rely on high-density LiDAR is a critical research problem because of the expense of LiDAR systems compared to cameras and other sensors. Recent research …
Autonomous VehiclesBird's-Eye View Semantic SegmentationData AugmentationNavigating the energy trilemma during geopolitical and environmental crises
There are many indicators of energy security. Few measure what really matters -- affordable and reliable energy supply -- and the trade-offs between the two. Reliability is physical, affordability is economic. Russia's l…
What Really is Deep Learning Doing?
Deep learning has achieved a great success in many areas, from computer vision to natural language processing, to game playing, and much more. Yet, what deep learning is really doing is still an open question. There are …
Deep LearningOpen-Ended Question AnsweringCorporate core values and social responsibility: What really matters to whom
This study uses an innovative measure, the Semantic Brand Score, to assess the interest of stakeholders in different company core values. Among others, we focus on corporate social responsibility (CSR) core value stateme…
Does Verbose Chain-of-Thought Really Help? In-Distribution Evidence that Content, Not Length, Matters
Chain-of-thought (CoT) prompting improves LLM reasoning, but the source is contested: do the intermediate steps help because they carry useful semantic content, or because conditioning on more tokens buys extra computati…