Evaluating saliency methods on artificial data with different background types
Over the last years, many 'explainable artificial intelligence' (xAI) approaches have been developed, but these have not always been objectively evaluated. To evaluate the quality of heatmaps generated by various saliency methods, we developed a framework to generate artificial data with synthetic lesions and a known ground truth map. Using this framework, we evaluated two data sets with different backgrounds, Perlin noise and 2D brain MRI slices, and found that the heatmaps vary strongly between saliency methods and backgrounds. We strongly encourage further evaluation of saliency maps and xAI methods using this framework before applying these in clinical or other safety-critical settings.
Code (1)
Tasks
Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)Similar Papers 제목 키워드 기반
Evaluating Input Perturbation Methods for Interpreting CNNs and Saliency Map Comparison
Input perturbation methods occlude parts of an input to a function and measure the change in the function's output. Recently, input perturbation methods have been applied to generate and evaluate saliency maps from convo…
Evaluating Saliency Methods for Neural Language Models
Saliency methods are widely used to interpret neural network predictions, but different variants of saliency methods often disagree even on the interpretations of the same prediction made by the same model. In these case…
SentenceRevisiting Sanity Checks for Saliency Maps
Saliency methods are a popular approach for model debugging and explainability. However, in the absence of ground-truth data for what the correct maps should be, evaluating and comparing different approaches remains a lo…
Benchmarking Perturbation-based Saliency Maps for Explaining Atari Agents
One of the most prominent methods for explaining the behavior of Deep Reinforcement Learning (DRL) agents is the generation of saliency maps that show how much each pixel attributed to the agents' decision. However, ther…
Atari GamesBenchmarkingDeep Reinforcement LearningFeature Importance+1Decoding Layer Saliency in Language Transformers
In this paper, we introduce a strategy for identifying textual saliency in large-scale language models applied to classification tasks. In visual networks where saliency is more well-studied, saliency is naturally locali…