A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
Backpropagation-based visualizations have been proposed to interpret convolutional neural networks (CNNs), however a theory is missing to justify their behaviors: Guided backpropagation (GBP) and deconvolutional network (DeconvNet) generate more human-interpretable but less class-sensitive visualizations than saliency map. Motivated by this, we develop a theoretical explanation revealing that GBP and DeconvNet are essentially doing (partial) image recovery which is unrelated to the network decisions. Specifically, our analysis shows that the backward ReLU introduced by GBP and DeconvNet, and the local connections in CNNs are the two main causes of compelling visualizations. Extensive experiments are provided that support the theoretical analysis.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unifying Perplexing Behaviors in Modified BP Attributions through Alignment Perspective
Attributions aim to identify input pixels that are relevant to the decision-making process. A popular approach involves using modified backpropagation (BP) rules to reverse decisions, which improves interpretability comp…
Decision MakingSensitivityTowards Predictable Feature Attribution: Revisiting and Improving Guided BackPropagation
Recently, backpropagation(BP)-based feature attribution methods have been widely adopted to interpret the internal mechanisms of convolutional neural networks (CNNs), and expected to be human-understandable (lucidity) an…
Decision MakingA Rigorous Study Of The Deep Taylor Decomposition
Saliency methods attempt to explain deep neural networks by highlighting the most salient features of a sample. Some widely used methods are based on a theoretical framework called Deep Taylor Decomposition (DTD), which …
Understanding Individual Decisions of CNNs via Contrastive Backpropagation
A number of backpropagation-based approaches such as DeConvNets, vanilla Gradient Visualization and Guided Backpropagation have been proposed to better understand individual decisions of deep convolutional neural network…
General ClassificationAdversarial Examples Are Not Bugs, They Are Superposition
Adversarial examples -- inputs with imperceptible perturbations that fool neural networks -- remain one of deep learning's most perplexing phenomena despite nearly a decade of research. While numerous defenses and explan…