Minionese: Comprehensive Benchmark and Mechanistic Study of Multilingual LLM Safety
Safety alignment in large language models remains brittle across languages: prompts reliably refused in English can elicit harmful compliance in non-English and low-resource settings. We introduce \textsc{Minionese}, a multilingual jailbreak benchmark spanning 18 languages, 4 resource tiers, and 4 perturbation types (standard translation, code-switching, transliteration, and translationese), paired with a geometric mechanistic analysis of refusal failure across language tiers. We show that each attack type produces a distinct vulnerability profile: transliteration vulnerability is mediated by script identity, code-switching maintains effectiveness through the lowest-resource tier, and a sharp safety regime transition between Tiers 2 and 3 is consistent across all models. Mechanistically, low-resource jailbreaks succeed by routing harmful content through a geometrically misaligned subspace that projects insufficiently onto the refusal directions, leaving the refusal mechanism intact but untriggered. These findings show that English-only safety evaluations are insufficient; they require accounting for script family, perturbation type, and per-language alignment coverage. The benchmark and analysis code is at https://github.com/Brentkong/Minionese-Comprehensive-Benchmark-and-Mechanistic-Study-of-Multilingual-LLM-Safety.git.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models
Language confusion -- where large language models (LLMs) generate unintended languages against the user's need -- remains a critical challenge, especially for English-centric models. We present the first mechanistic inte…
BenchmarkingLanguage ModelingLanguage ModellingGetting More from Less: Large Language Models are Good Spontaneous Multilingual Learners
Recently, Large Language Models (LLMs) have shown impressive language capabilities. While most of the existing LLMs have very unbalanced performance across different languages, multilingual alignment based on translation…
TranslationEMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models
In this work, we introduce EMMA-500, a large-scale multilingual language model continue-trained on texts across 546 languages designed for enhanced multilingual performance, focusing on improving language coverage for lo…
Cross-Lingual TransferLanguage ModelingLanguage ModellingMultilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
Sparse autoencoders (SAEs) enable feature-level mechanistic interpretability and activation steering in large language models (LLMs), but SAE-based language control remains unreliable in multilingual settings: most SAEs …
Language IdentificationMachine TranslationBeyond English-Centric Training: How Reinforcement Learning Improves Cross-Lingual Reasoning in LLMs
Enhancing the complex reasoning capabilities of Large Language Models (LLMs) attracts widespread attention. While reinforcement learning (RL) has shown superior performance for improving complex reasoning, its impact on …
Reinforcement Learning