Explaining with Counter Visual Attributes and Examples
In this paper, we aim to explain the decisions of neural networks by utilizing multimodal information. That is counter-intuitive attributes and counter visual examples which appear when perturbed samples are introduced. Different from previous work on interpreting decisions using saliency maps, text, or visual patches we propose to use attributes and counter-attributes, and examples and counter-examples as part of the visual explanations. When humans explain visual decisions they tend to do so by providing attributes and examples. Hence, inspired by the way of human explanations in this paper we provide attribute-based and example-based explanations. Moreover, humans also tend to explain their visual decisions by adding counter-attributes and counter-examples to explain what is not seen. We introduce directed perturbations in the examples to observe which attribute values change when classifying the examples into the counter classes. This delivers intuitive counter-attributes and counter-examples. Our experiments with both coarse and fine-grained datasets show that attributes provide discriminating and human-understandable intuitive and counter-intuitive explanations.
Code (1)
Tasks
AttributeSimilar Papers 제목 키워드 기반
Explaining Deep Learning Models with Constrained Adversarial Examples
Machine learning algorithms generally suffer from a problem of explainability. Given a classification result from a model, it is typically hard to determine what caused the decision to be made, and to give an informative…
ClassificationcounterfactualDeep LearningGeneral ClassificationOn Quantitative Evaluations of Counterfactuals
As counterfactual examples become increasingly popular for explaining decisions of deep learning models, it is essential to understand what properties quantitative evaluation metrics do capture and equally important what…
counterfactualComparables XAI: Faithful Example-based AI Explanations with Counterfactual Trace Adjustments
Explaining with examples is an intuitive way to justify AI decisions. However, it is challenging to understand how a decision value should change relative to the examples with many features differing by large amounts. We…
Retrieving Counterfactuals Improves Visual In-Context Learning
Vision-language models (VLMs) have achieved impressive performance across a wide range of multimodal reasoning tasks, but they often struggle to disentangle fine-grained visual attributes and reason about underlying caus…
Multimodal ReasoningImage RetrievalSim2Word: Explaining Similarity with Representative Attribute Words via Counterfactual Explanations
Recently, we have witnessed substantial success using the deep neural network in many tasks. While there still exists concerns about the explainability of decision-making, it is beneficial for users to discern the defect…
AttributecounterfactualCounterfactual ExplanationDecision Making+4