Uncertainty based Class Activation Maps for Visual Question Answering
Understanding and explaining deep learning models is an imperative task. Towards this, we propose a method that obtains gradient-based certainty estimates that also provide visual attention maps. Particularly, we solve for visual question answering task. We incorporate modern probabilistic deep learning methods that we further improve by using the gradients for these estimates. These have two-fold benefits: a) improvement in obtaining the certainty estimates that correlate better with misclassified samples and b) improved attention maps that provide state-of-the-art results in terms of correlation with human attention regions. The improved attention maps result in consistent improvement for various methods for visual question answering. Therefore, the proposed technique can be thought of as a recipe for obtaining improved certainty estimates and explanations for deep learning models. We provide detailed empirical analysis for the visual question answering task on all standard benchmarks and comparison with state of the art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningProbabilistic Deep LearningQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
U-CAM: Visual Explanation using Uncertainty based Class Activation Maps
Understanding and explaining deep learning models is an imperative task. Towards this, we propose a method that obtains gradient-based certainty estimates that also provide visual attention maps. Particularly, we solve f…
Deep LearningProbabilistic Deep LearningQuestion AnsweringVisual Question Answering+1Visualizing Uncertainty: Spatial Maps of Missing and Conflicting Evidence in Deep Learning
Understanding when and why deep neural networks are uncertain is crucial for deploying reliable machine learning systems in safety-critical domains. While existing uncertainty quantification methods provide scalar measur…
GAM: Explainable Visual Similarity and Classification via Gradient Activation Maps
We present Gradient Activation Maps (GAM) - a machinery for explaining predictions made by visual similarity and classification models. By gleaning localized gradient and activation information from multiple network laye…
ClassificationOnline Refinement of Low-level Feature Based Activation Map for Weakly Supervised Object Localization
We present a two-stage learning framework for weakly supervised object localization (WSOL). While most previous efforts rely on high-level feature based CAMs (Class Activation Maps), this paper proposes to localize objec…
ObjectObject LocalizationWeakly-Supervised Object LocalizationCombinational Class Activation Maps for Weakly Supervised Object Localization
Weakly supervised object localization has recently attracted attention since it aims to identify both class labels and locations of objects by using image-level labels. Most previous methods utilize the activation map co…
ObjectObject LocalizationWeakly-Supervised Object Localization