AMIA: Automatic Masking and Joint Intention Analysis Makes LVLMs Robust Jailbreak Defenders
We introduce AMIA, a lightweight, inference-only defense for Large Vision-Language Models (LVLMs) that (1) Automatically Masks a small set of text-irrelevant image patches to disrupt adversarial perturbations, and (2) conducts joint Intention Analysis to uncover and mitigate hidden harmful intents before response generation. Without any retraining, AMIA improves defense success rates across diverse LVLMs and jailbreak benchmarks from an average of 52.4% to 81.7%, preserves general utility with only a 2% average accuracy drop, and incurs only modest inference overhead. Ablation confirms both masking and intention analysis are essential for a robust safety-utility trade-off.
Code (0)
등록된 구현이 없습니다.
Tasks
Response GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ModalImmune: Immunity Driven Unlearning via Self Destructive Training
Multimodal systems are vulnerable to partial or complete loss of input channels at deployment, which undermines reliability in real-world settings. This paper presents ModalImmune, a training framework that enforces moda…
Resolving issues of scaling for gramian based input-output pairing methods
A key problem in process control is to decide which inputs should control which outputs. There are multiple ways to solve this problem, among them using gramian based measures, which include the Hankel interaction index …
Automatic Identification of Cuneiform Fragments Using String Alignment Algorithms
The literature from ancient Mesopotamia is still riddled with textual lacunas. Scores of fragments which could potentially fill those lacunas lie unidentified in museums's cabinets, but their identification has tradition…
PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining
We present PANORAMIA, a privacy leakage measurement framework for machine learning models that relies on membership inference attacks using generated data as non-members. By relying on generated non-member data, PANORAMI…
Robust Automatic Differentiation of Square-Root Kalman Filters via Gramian Differentials
Square-root Kalman filters propagate state covariances in Cholesky-factor form for numerical stability, and are a natural target for gradient-based parameter learning in state-space models. Their core operation, triangul…