Fooling Network Interpretation in Image Classification
Deep neural networks have been shown to be fooled rather easily using adversarial attack algorithms. Practical methods such as adversarial patches have been shown to be extremely effective in causing misclassification. However, these patches are highlighted using standard network interpretation algorithms, thus revealing the identity of the adversary. We show that it is possible to create adversarial patches which not only fool the prediction, but also change what we interpret regarding the cause of the prediction. Moreover, we introduce our attack as a controlled setting to measure the accuracy of interpretation algorithms. We show this using extensive experiments for Grad-CAM interpretation that transfers to occluding patch interpretation as well. We believe our algorithms can facilitate developing more robust network interpretation tools that truly explain the network's underlying decision making process.
Code (0)
등록된 구현이 없습니다.
Tasks
Adversarial AttackClassificationDecision MakingGeneral Classificationimage-classificationImage ClassificationNetwork InterpretationSimilar Papers 제목 키워드 기반
Fooling Neural Network Interpretations via Adversarial Model Manipulation
We ask whether the neural network interpretation methods can be fooled via adversarial model manipulation, which is defined as a model fine-tuning step that aims to radically alter the explanations without hurting the ac…
modelNetwork InterpretationSPOOF: Simple Pixel Operations for Out-of-Distribution Fooling
Deep neural networks (DNNs) excel across image recognition tasks, yet continue to exhibit overconfidence on inputs that bear no resemblance to natural images. Revisiting the "fooling images" work introduced by Nguyen et …
Recognition Awareness: An Application of Latent Cognizance to Open-Set Recognition
This study investigates an application of a new probabilistic interpretation of a softmax output to Open-Set Recognition (OSR). Softmax is a mechanism wildly used in classification and object recognition. However, a soft…
ObjectObject RecognitionOpen Set LearningFooBaR: Fault Fooling Backdoor Attack on Neural Network Training
Neural network implementations are known to be vulnerable to physical attack vectors such as fault injection attacks. As of now, these attacks were only utilized during the inference phase with the intention to cause a m…
Backdoor Attackimage-classificationImage ClassificationArt of singular vectors and universal adversarial perturbations
Vulnerability of Deep Neural Networks (DNNs) to adversarial attacks has been attracting a lot of attention in recent studies. It has been shown that for many state of the art DNNs performing image classification there ex…
image-classificationImage Classification