Quantifying Explainability of Saliency Methods in Deep Neural Networks with a Synthetic Dataset
Post-hoc analysis is a popular category in eXplainable artificial intelligence (XAI) study. In particular, methods that generate heatmaps have been used to explain the deep neural network (DNN), a black-box model. Heatmaps can be appealing due to the intuitive and visual ways to understand them but assessing their qualities might not be straightforward. Different ways to assess heatmaps' quality have their own merits and shortcomings. This paper introduces a synthetic dataset that can be generated adhoc along with the ground-truth heatmaps for more objective quantitative assessment. Each sample data is an image of a cell with easily recognized features that are distinguished from localization ground-truth mask, hence facilitating a more transparent assessment of different XAI methods. Comparison and recommendations are made, shortcomings are clarified along with suggestions for future research directions to handle the finer details of select post-hoc analysis methods. Furthermore, mabCAM is introduced as the heatmap generation method compatible with our ground-truth heatmaps. The framework is easily generalizable and uses only standard deep learning components.
Code (2)
Tasks
Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)Similar Papers 제목 키워드 기반
Learning Temporal Saliency for Time Series Forecasting with Cross-Scale Attention
Explainability in time series forecasting is essential for improving model transparency and supporting informed decision-making. In this work, we present CrossScaleNet, an innovative architecture that combines a patch-ba…
Time Series ForecastingFeature ImportanceSaliency DetectionHow Can One Choose the Best CAM-Based Explainability Method for a CNN Model?
In recent years, several advances have been observed in Deep Learning with surprising results. Models in this area have been increasingly used in numerous applications, including those sensitive to human life, which requ…
CNN Explainability with Multivector Tucker Saliency Maps for Self-Supervised Models
Interpreting the decisions of Convolutional Neural Networks (CNNs) is essential for understanding their behavior, yet explainability remains a significant challenge, particularly for self-supervised models. Most existing…
Tensor DecompositionEvaluation of Saliency-based Explainability Method
A particular class of Explainable AI (XAI) methods provide saliency maps to highlight part of the image a Convolutional Neural Network (CNN) model looks at to classify the image as a way to explain its working. These met…
Explainable Artificial Intelligence (XAI)Global Saliency: Aggregating Saliency Maps to Assess Dataset Artefact Bias
In high-stakes applications of machine learning models, interpretability methods provide guarantees that models are right for the right reasons. In medical imaging, saliency maps have become the standard tool for determi…
Semantic Segmentation