Reconstructing Actions To Explain Deep Reinforcement Learning
Feature attribution has been a foundational building block for explaining the input feature importance in supervised learning with Deep Neural Network (DNNs), but face new challenges when applied to deep Reinforcement Learning (RL).We propose a new approach to explaining deep RL actions by defining a class of \emph{action reconstruction} functions that mimic the behavior of a network in deep RL. This approach allows us to answer more complex explainability questions than direct application of DNN attribution methods, which we adapt to \emph{behavior-level attributions} in building our action reconstructions. It also allows us to define \emph{agreement}, a metric for quantitatively evaluating the explainability of our methods. Our experiments on a variety of Atari games suggest that perturbation-based attribution methods are significantly more suitable in reconstructing actions to explain the deep RL agent than alternative attribution methods, and show greater \emph{agreement} than existing explainability work utilizing attention. We further show that action reconstruction allows us to demonstrate how a deep agent learns to play Pac-Man game.
Code (0)
등록된 구현이 없습니다.
Tasks
Atari GamesDeep Reinforcement LearningFeature Importancereinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Explainable Reinforcement Learning via a Causal World Model
Generating explanations for reinforcement learning (RL) is challenging as actions may produce long-term effects on the future. In this paper, we develop a novel framework for explainable RL by learning a causal world mod…
modelreinforcement-learningReinforcement LearningReinforcement Learning (RL)Explaining Agent's Decision-making in a Hierarchical Reinforcement Learning Scenario
Reinforcement learning is a machine learning approach based on behavioral psychology. It is focused on learning agents that can acquire knowledge and learn to carry out new tasks by interacting with the environment. Howe…
Decision MakingHierarchical Reinforcement Learningreinforcement-learningReinforcement Learning+1BXRL: Behavior-Explainable Reinforcement Learning
A major challenge of Reinforcement Learning is that agents often learn undesired behaviors that seem to defy the reward structure they were given. Explainable Reinforcement Learning (XRL) methods can answer queries such …
Reinforcement LearningInherently Explainable Reinforcement Learning in Natural Language
We focus on the task of creating a reinforcement learning agent that is inherently explainable -- with the ability to produce immediate local explanations by thinking out loud while performing a task and analyzing entire…
Graph Attentionreinforcement-learningReinforcement LearningReinforcement Learning (RL)Inherently Explainable Reinforcement Learning in Natural Language
We focus on the task of creating a reinforcement learning agent that is inherently explainable---with the ability to produce immediate local explanations by thinking out loud while performing a task and analyzing entire …
Graph Attentionreinforcement-learningReinforcement LearningReinforcement Learning (RL)