Attentive Explanations: Justifying Decisions and Pointing to the Evidence (Extended Abstract)
Deep models are the defacto standard in visual decision problems due to their impressive performance on a wide array of visual tasks. On the other hand, their opaqueness has led to a surge of interest in explainable systems. In this work, we emphasize the importance of model explanation in various forms such as visual pointing and textual justification. The lack of data with justification annotations is one of the bottlenecks of generating multimodal explanations. Thus, we propose two large-scale datasets with annotations that visually and textually justify a classification decision for various activities, i.e. ACT-X, and for question answering, i.e. VQA-X. We also introduce a multimodal methodology for generating visual and textual explanations simultaneously. We quantitatively show that training with the textual explanations not only yields better textual justification models, but also models that better localize the evidence that support their decision.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Attentive Explanations: Justifying Decisions and Pointing to the Evidence
Deep models are the defacto standard in visual decision models due to their impressive performance on a wide array of visual tasks. However, they are frequently seen as opaque and are unable to explain their decisions. I…
Decision MakingQuestion AnsweringSentenceVisual Question Answering+1Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
Deep models that are both effective and explainable are desirable in many settings; prior explainable models have been unimodal, offering either image-based visualization of attention weights or text-based generation of …
Activity RecognitionExplainable ModelsQuestion AnsweringVisual Question Answering+1Where and When: Space-Time Attention for Audio-Visual Explanations
Explaining the decision of a multi-modal decision-maker requires to determine the evidence from both modalities. Recent advances in XAI provide explanations for models trained on still images. However, when it comes to m…
Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)Multimodal Deep LearningBeyond Verification: Abductive Explanations for Post-AI Assessment of Privacy Leakage
Privacy leakage in AI-based decision processes poses significant risks, particularly when sensitive information can be inferred. We propose a formal framework to audit privacy leakage using abductive explanations, which …
Explainable Decision Making with Lean and Argumentative Explanations
It is widely acknowledged that transparency of automated decision making is crucial for deployability of intelligent systems, and explaining the reasons why some decisions are "good" and some are not is a way to achievin…
Decision Making