On the Connection Between Adversarial Robustness and Saliency Map Interpretability
Recent studies on the adversarial vulnerability of neural networks have shown that models trained to be more robust to adversarial attacks exhibit more interpretable saliency maps than their non-robust counterparts. We aim to quantify this behavior by considering the alignment between input image and saliency map. We hypothesize that as the distance to the decision boundary grows,so does the alignment. This connection is strictly true in the case of linear models. We confirm these theoretical findings with experiments based on models trained with a local Lipschitz regularization and identify where the non-linear nature of neural networks weakens the relation.
Code (1)
Tasks
Adversarial RobustnessSimilar Papers 제목 키워드 기반
On Saliency Maps and Adversarial Robustness
A Very recent trend has emerged to couple the notion of interpretability and adversarial robustness, unlike earlier efforts which solely focused on good interpretations or robustness against adversaries. Works have shown…
Adversarial RobustnessExploring the Interplay of Interpretability and Robustness in Deep Neural Networks: A Saliency-guided Approach
Adversarial attacks pose a significant challenge to deploying deep learning models in safety-critical applications. Maintaining model robustness while ensuring interpretability is vital for fostering trust and comprehens…
Decision MakingImproving the Interpretability of fMRI Decoding using Deep Neural Networks and Adversarial Robustness
Deep neural networks (DNNs) are being increasingly used to make predictions from functional magnetic resonance imaging (fMRI) data. However, they are widely seen as uninterpretable "black boxes", as it can be difficult t…
Adversarial RobustnessPolicy Distillation with Selective Input Gradient Regularization for Efficient Interpretability
Although deep Reinforcement Learning (RL) has proven successful in a wide range of tasks, one challenge it faces is interpretability when applied to real-world problems. Saliency maps are frequently used to provide inter…
Autonomous DrivingDeep Reinforcement LearningReinforcement Learning (RL)SCAAT: Improving Neural Network Interpretability via Saliency Constrained Adaptive Adversarial Training
Deep Neural Networks (DNNs) are expected to provide explanation for users to understand their black-box predictions. Saliency map is a common form of explanation illustrating the heatmap of feature attributions, but it s…