A Note about: Local Explanation Methods for Deep Neural Networks lack Sensitivity to Parameter Values
Local explanation methods, also known as attribution methods, attribute a deep network's prediction to its input (cf. Baehrens et al. (2010)). We respond to the claim from Adebayo et al. (2018) that local explanation methods lack sensitivity, i.e., DNNs with randomly-initialized weights produce explanations that are both visually and quantitatively similar to those produced by DNNs with learned weights. Further investigation reveals that their findings are due to two choices in their analysis: (a) ignoring the signs of the attributions; and (b) for integrated gradients (IG), including pixels in their analysis that have zero attributions by choice of the baseline (an auxiliary input relative to which the attributions are computed). When both factors are accounted for, IG attributions for a random network and the actual network are uncorrelated. Our investigation also sheds light on how these issues affect visualizations, although we note that more work is needed to understand how viewers interpret the difference between the random and the actual attributions.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeSensitivitySimilar Papers 제목 키워드 기반
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges
Issues regarding explainable AI involve four components: users, laws & regulations, explanations and algorithms. Together these components provide a context in which explanation methods can be evaluated regarding their a…
Deep LearningLocal Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values
Explaining the output of a complicated machine learning model like a deep neural network (DNN) is a central challenge in machine learning. Several proposed local explanation methods address this issue by identifying what…
BIG-bench Machine LearningSensitivityDiConStruct: Causal Concept-based Explanations through Black-Box Distillation
Model interpretability plays a central role in human-AI decision-making systems. Ideally, explanations should be expressed using human-interpretable semantic concepts. Moreover, the causal relations between these concept…
Decision MakingCan local explanation techniques explain linear additive models?
Local model-agnostic additive explanation techniques decompose the predicted output of a black-box model into additive feature importance scores. Questions have been raised about the accuracy of the produced local additi…
Additive modelsExplanation Fidelity EvaluationFeature ImportanceGLocalX -- From Local to Global Explanations of Black Box AI Models
Artificial Intelligence (AI) has come to prominence as one of the major components of our society, with applications in most aspects of our lives. In this field, complex and highly nonlinear machine learning models such …
Decision Making