Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?
Large language models (LLMs) undergo safety alignment after training and tuning, yet recent work shows that safety can be bypassed through jailbreak attacks. While many jailbreaks and defenses exist, their cross-lingual generalization remains underexplored. This paper presents the first systematic multilingual evaluation of jailbreaks and defenses across ten languages -- spanning high-, medium-, and low-resource languages -- using six LLMs on HarmBench and AdvBench. We assess two jailbreak types: logical-expression-based and adversarial-prompt-based. For both types, attack success and defense robustness vary across languages: high-resource languages are safer under standard queries but more vulnerable to adversarial ones. Simple defenses can be effective, but are language- and model-dependent. These findings call for language-aware and cross-lingual safety benchmarks for LLMs.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Defending Jailbreak Prompts via In-Context Adversarial Game
Large Language Models (LLMs) demonstrate remarkable capabilities across diverse applications. However, concerns regarding their security, particularly the vulnerability to jailbreak attacks, persist. Drawing inspiration …
No Free Lunch for Defending Against Prefilling Attack by In-Context Learning
The security of Large Language Models (LLMs) has become an important research topic since the emergence of ChatGPT. Though there have been various effective methods to defend against jailbreak attacks, prefilling attacks…
In-Context LearningSafety AlignmentSentenceDefending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
Large language models (LLMs) are increasingly being adopted in a wide range of real-world applications. Despite their impressive performance, recent studies have shown that LLMs are vulnerable to deliberately crafted adv…
EEG-Defender: Defending against Jailbreak through Early Exit Generation of Large Language Models
Large Language Models (LLMs) are increasingly attracting attention in various applications. Nonetheless, there is a growing concern as some users attempt to exploit these models for malicious purposes, including the synt…
EEGPrompt EngineeringLLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper
Jailbreaking is an emerging adversarial attack that bypasses the safety alignment deployed in off-the-shelf large language models (LLMs). A considerable amount of research exists proposing more effective jailbreak attack…
Adversarial AttackSafety Alignment