Can Rationalization Improve Robustness?
A growing line of work has investigated the development of neural NLP models that can produce rationales---subsets of input that can explain their model predictions.
In this paper, we ask whether such rationale models can also provide robustness to adversarial attacks in addition to their interpretable nature. Since these models need to first generate rationales (`rationalizer'') before making predictions (predictor''), they have the potential to ignore noise or adversarially added text by simply masking it out of the generated rationale. To this end, we systematically generate various types of AddText' attacks for both token and sentence-level rationalization tasks, and perform an extensive empirical evaluation of state-of-the-art rationale models across five different tasks.
Our experiments reveal that the rationale models show the promise to improve robustness, while they struggle in certain scenarios---when the rationalizer is sensitive to positional bias or lexical choices of attack text. Further, leveraging human rationale as supervision does not always translate to better performance. Our study is a first step towards exploring the interplay between interpretability and robustness in the rationalize-then-predict framework.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSimilar Papers 제목 키워드 기반
Does Self-Rationalization Improve Robustness to Spurious Correlations?
Rationalization is fundamental to human reasoning and learning. NLP models trained to produce rationales along with predictions, called self-rationalization models, have been investigated for their interpretability and u…
DecoderAdversarial Attack for Explanation Robustness of Rationalization Models
Rationalization models, which select a subset of input text as rationale-crucial for humans to understand and trust predictions-have recently emerged as a prominent research area in eXplainable Artificial Intelligence. H…
Adversarial AttackExplainable artificial intelligenceCREST: A Joint Framework for Rationalization and Counterfactual Text Generation
Selective rationales and counterfactual examples have emerged as two effective, complementary classes of interpretability methods for analyzing and training NLP models. However, prior work has not explored how these meth…
counterfactualData AugmentationText GenerationvalidCan Rationalization Improve Robustness?
A growing line of work has investigated the development of neural NLP models that can produce rationales--subsets of input that can explain their model predictions. In this paper, we ask whether such rationale models can…
SentenceRationalization for Explainable NLP: A Survey
Recent advances in deep learning have improved the performance of many Natural Language Processing (NLP) tasks such as translation, question-answering, and text classification. However, this improvement comes at the expe…
Explainable Artificial Intelligence (XAI)Question AnsweringSurveytext-classification+2