Towards Predictable Feature Attribution: Revisiting and Improving Guided BackPropagation
Recently, backpropagation(BP)-based feature attribution methods have been widely adopted to interpret the internal mechanisms of convolutional neural networks (CNNs), and expected to be human-understandable (lucidity) and faithful to decision-making processes (fidelity). In this paper, we introduce a novel property for feature attribution: predictability, which means users can forecast behaviors of the interpretation methods. With the evidence that many attribution methods have unexpected and harmful phenomena like class-insensitivity, the predictability is critical to avoid over-trust and misuse from users. Observing that many intuitive improvements for lucidity and fidelity tend to sacrifice predictability, we propose a new visual explanation method called TR-GBP (Theoretical Refinements of Guided BackPropagation) which revisits and improves GBP from theoretical perspective rather than solely optimizing the attribution performance. Qualitative and quantitative experiments show that TR-GBP is more visually sharpened, gets rid of the fidelity problems in GBP, and effectively predicts the possible behaviors so that we can easily discriminate some prediction errors from interpretation errors. The codes of TR-GBP are available in supplementary and will be open source.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision MakingSimilar Papers 제목 키워드 기반
A General Taylor Framework for Unifying and Revisiting Attribution Methods
Attribution methods provide an insight into the decision-making process of machine learning models, especially deep neural networks, by assigning contribution scores to each individual feature. However, the attribution p…
BenchmarkingDecision MakingTesting the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer's disease classification
Attribution methods are an easy to use tool for investigating and validating machine learning models. Multiple methods have been suggested in the literature and it is not yet clear which method is most suitable for a giv…
General ClassificationImproving Deep Learning Interpretability by Saliency Guided Training
Saliency methods have been widely used to highlight important input features in model predictions. Most existing methods use backpropagation on a modified gradient function to generate saliency maps. Thus, noisy gradient…
Deep LearningTime SeriesTime Series AnalysisGradient Backpropagation based Feature Attribution to Enable Explainable-AI on the Edge
There has been a recent surge in the field of Explainable AI (XAI) which tackles the problem of providing insights into the behavior of black-box machine learning models. Within this field, \textit{feature attribution} e…
Explainable Artificial Intelligence (XAI)High-Level SynthesisBias Also Matters: Bias Attribution for Deep Neural Network Explanation
The gradient of a deep neural network (DNN) w.r.t. the input provides information that can be used to explain the output prediction in terms of the input features and has been widely studied to assist in interpreting DNN…
Attribute