Full-Gradient Representation for Neural Network Visualization
We introduce a new tool for interpreting neural net responses, namely full-gradients, which decomposes the neural net response into input sensitivity and per-neuron sensitivity components. This is the first proposed representation which satisfies two key properties: completeness and weak dependence, which provably cannot be satisfied by any saliency map-based interpretability method. For convolutional nets, we also propose an approximate saliency map representation, called FullGrad, obtained by aggregating the full-gradient components. We experimentally evaluate the usefulness of FullGrad in explaining model behaviour with two quantitative tests: pixel perturbation and remove-and-retrain. Our experiments reveal that our method explains model behaviour correctly, and more comprehensively than other methods in the literature. Visual inspection also reveals that our saliency maps are sharper and more tightly confined to object regions than other methods.
Code (2)
Tasks
Interpretable Machine LearningSensitivityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Perception Visualization: Seeing Through the Eyes of a DNN
Artificial intelligence (AI) systems power the world we live in. Deep neural networks (DNNs) are able to solve tasks in an ever-expanding landscape of scenarios, but our eagerness to apply these powerful models leads us …
Attention Guided CAM: Visual Explanations of Vision Transformer Guided by Self-Attention
Vision Transformer(ViT) is one of the most widely used models in the computer vision field with its great performance on various tasks. In order to fully utilize the ViT-based architecture in various applications, proper…
Grad-CAM: Why did you say that?
We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations. Our approach, called Gradien…
Image CaptioningVisual Question AnsweringVisual Question Answering (VQA)Explaining Convolutional Neural Networks using Softmax Gradient Layer-wise Relevance Propagation
Convolutional Neural Networks (CNN) have become state-of-the-art in the field of image classification. However, not everything is understood about their inner representations. This paper tackles the interpretability and …
AttributeClassificationGeneral Classificationimage-classification+2Variational Transfer Learning using Cross-Domain Latent Modulation
To successfully apply trained neural network models to new domains, powerful transfer learning solutions are essential. We propose to introduce a novel cross-domain latent modulation mechanism to a variational autoencode…
Domain AdaptationImage-to-Image TranslationTransfer LearningUnsupervised Domain Adaptation