paper-with-me

Papers

JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models

2026-05-27 · Jiachen Qian arxiv

Forensic vision-language models (VLMs) have recently been developed to detect image tampering and provide natural-language explanations. However, their robustness against adversarial manipulation remains underexplored. Existing adversarial attacks typically aim to flip the model's binary judgment, while the accompanying explanation may still reveal forensic cues and contradict the attacked judgment. In this paper, we study judgment-explanation consistent adversarial attacks against forensic VLMs and propose JECA^2, a controlled white-box red-team diagnostic that jointly redirects visual attribution and aligns textual explanations with the target judgment. On the visual side, JECA^2 uses Grad-CAM-guided perturbations to divert attribution from tampered regions toward benign regions. On the textual side, it optimizes prompt embeddings toward authenticity-affirming semantics under a token-proximity constraint. Experiments on forensic VLM benchmarks show that JECA^2 achieves higher attack success and automated judgment-explanation consistency than implemented baselines under white-box threat settings, while transfer to closed-source VLMs remains measurable but limited. Our results highlight a consistency failure mode in explanation-based forensic VLMs and motivate future robustness evaluation beyond binary detection accuracy.

📄 PDF Abstract BibTeX arXiv:2605.28609

Code (0)

등록된 구현이 없습니다.

Tasks

Adversarial Attack

Similar Papers 제목 키워드 기반

InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents

2024-03-05 · Qiusi Zhan, Zhixiang Liang, Zifan Ying, Daniel Kang

Recent work has embodied LLMs as agents, allowing them to access tools, perform actions, and interact with external content (e.g., emails or websites). However, external content introduces the risk of indirect prompt inj…

BenchmarkingLanguage ModelingLanguage ModellingLarge Language Model

ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agents

2025-09-26 · Hwan Chang, Yonghyun Jun, Hwanhee Lee arxiv

The growing deployment of large language model (LLM) based agents that interact with external environments has created new attack surfaces for adversarial manipulation. One major threat is indirect prompt injection, wher…

Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations

2019-10-07 · ACL 2020 6 · Oana-Maria Camburu, Brendan Shillingford, Pasquale Minervini, Thomas Lukasiewicz 외

To increase trust in artificial intelligence systems, a promising research direction consists of designing neural models capable of generating natural language explanations for their predictions. In this work, we show th…

Decision MakingNatural Language Inference

Exposing Vulnerabilities in Explanation for Time Series Classifiers via Dual-Target Attacks

2026-02-02 · Bohan Wang, Zewen Liu, Lu Lin, Hui Liu 외 arxiv

Interpretable time series deep learning systems are often assessed by checking temporal consistency on explanations, implicitly treating this as evidence of robustness. We show that this assumption can fail: Predictions …

Right Predictions, Misleading Explanations: On the Vulnerability of Vision-Language Model Explanations

2026-05-15 · Narges Babadi, Hadis Karimipour arxiv

Explanation mechanisms are increasingly used to support transparency and trust in vision-language models (VLMs), particularly in settings where model decisions require human oversight. However, the robustness of these ex…