Are Gradient-based Saliency Maps Useful in Deep Reinforcement Learning?

2020-12-02 · NeurIPS Workshop ICBINB 2020 12 · Matthias Rosynski, Frank Kirchner, Matias Valdenegro-Toro

Deep Reinforcement Learning (DRL) connects the classic Reinforcement Learning algorithms with Deep Neural Networks. A problem in DRL is that CNNs are black-boxes and it is hard to understand the decision-making process of agents. In order to be able to use RL agents in highly dangerous environments for humans and machines, the developer needs a debugging tool to assure that the agent does what is expected. Currently, rewards are primarily used to interpret how well an agent is learning. However, this can lead to deceptive conclusions if the agent receives more rewards by memorizing a policy and not learning to respond to the environment. In this work, it is shown that this problem can be recognized with the help of gradient visualization techniques. This work brings some of the best-known visualization methods from the field of image classification to the area of Deep Reinforcement Learning. Furthermore, two new visualization techniques have been developed, one of which provides particularly good results. It is being proven to what extent the algorithms can be used in the area of Reinforcement learning. Also, the question arises on how well the DRL algorithms can be visualized across different environments with varying visualization techniques.

📄 PDF Abstract BibTeX arXiv:2012.01281

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingDeep Reinforcement Learningimage-classification Image Classification reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Training for Trustworthy Saliency Maps: Adversarial Training Meets Feature-Map Smoothing

2026-03-07 · Dipkamal Bhusal, Md Tanvirul Alam, Nidhi Rastogi arxiv

Gradient-based saliency methods such as Vanilla Gradient (VG) and Integrated Gradients (IG) are widely used to explain image classifiers, yet the resulting maps are often noisy and unstable, limiting their usefulness in …

Attention-based Saliency Maps Improve Interpretability of Pneumothorax Classification

2023-03-03 · Alessandro Wollek, Robert Graf, Saša Čečatka, Nicola Fink 외

Purpose: To investigate chest radiograph (CXR) classification performance of vision transformers (ViT) and interpretability of attention-based saliency using the example of pneumothorax classification. Materials and Meth…

Classification Lung Disease Classification

Policy Distillation with Selective Input Gradient Regularization for Efficient Interpretability

2022-05-18 · Jinwei Xing, Takashi Nagata, Xinyun Zou, Emre Neftci 외

Although deep Reinforcement Learning (RL) has proven successful in a wide range of tasks, one challenge it faces is interpretability when applied to real-world problems. Saliency maps are frequently used to provide inter…

Autonomous Driving Deep Reinforcement Learning Reinforcement Learning (RL)

HiRA-CAM: Preserving Fine-Grained Spatial Relevance in Gradient-Based Visual Explanations

2026-08-19 · Manasi Nerurkar, Ali A. Minai arxiv

Deep Learning models can include billions of parameters or more, making it difficult to explain their internal transformations and outputs. However, explainability is increasing in importance due to the use of AI in cruc…

Visualizing and Understanding Atari Agents

2017-10-31 · ICML 2018 7 · Sam Greydanus, Anurag Koul, Jonathan Dodge, Alan Fern

While deep reinforcement learning (deep RL) agents are effective at maximizing rewards, it is often unclear what strategies they use to do so. In this paper, we take a step toward explaining deep RL agents through a case…

Deep Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)