Adversarial Attack for Explanation Robustness of Rationalization Models
Rationalization models, which select a subset of input text as rationale-crucial for humans to understand and trust predictions-have recently emerged as a prominent research area in eXplainable Artificial Intelligence. However, most of previous studies mainly focus on improving the quality of the rationale, ignoring its robustness to malicious attack. Specifically, whether the rationalization models can still generate high-quality rationale under the adversarial attack remains unknown. To explore this, this paper proposes UAT2E, which aims to undermine the explainability of rationalization models without altering their predictions, thereby eliciting distrust in these models from human users. UAT2E employs the gradient-based search on triggers and then inserts them into the original input to conduct both the non-target and target attack. Experimental results on five datasets reveal the vulnerability of rationalization models in terms of explanation, where they tend to select more meaningless tokens under attacks. Based on this, we make a series of recommendations for improving rationalization models in terms of explanation.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial AttackExplainable artificial intelligenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Can Rationalization Improve Robustness?
A growing line of work has investigated the development of neural NLP models that can produce rationales--subsets of input that can explain their model predictions. In this paper, we ask whether such rationale models can…
SentenceCan Rationalization Improve Robustness?
A growing line of work has investigated the development of neural NLP models that can produce rationales---subsets of input that can explain their model predictions. In this paper, we ask whether such rationale models ca…
SentenceOvercoming Adversarial Attacks for Human-in-the-Loop Applications
Including human analysis has the potential to positively affect the robustness of Deep Neural Networks and is relatively unexplored in the Adversarial Machine Learning literature. Neural network visual explanation maps h…
JECA^2: Judgment-Explanation Consistent Adversarial Attack against Forensic Vision-Language Models
Forensic vision-language models (VLMs) have recently been developed to detect image tampering and provide natural-language explanations. However, their robustness against adversarial manipulation remains underexplored. E…
Adversarial AttackRationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations
We introduce AI rationalization, an approach for generating explanations of autonomous system behavior as if a human had performed the behavior. We describe a rationalization technique that uses neural machine translatio…
Explanation GenerationMachine TranslationTranslation