EDGE: Explaining Deep Reinforcement Learning Policies

2021-12-01 · NeurIPS 2021 12 · Wenbo Guo, Xian Wu, Usmann Khan, Xinyu Xing

With the rapid development of deep reinforcement learning (DRL) techniques, there is an increasing need to understand and interpret DRL policies. While recent research has developed explanation methods to interpret how an agent determines its moves, they cannot capture the importance of actions/states to a game's final result. In this work, we propose a novel self-explainable model that augments a Gaussian process with a customized kernel function and an interpretable predictor. Together with the proposed model, we also develop a parameter learning procedure that leverages inducing points and variational inference to improve learning efficiency. Using our proposed model, we can predict an agent's final rewards from its game episodes and extract time step importance within episodes as strategy-level explanations for that agent. Through experiments on Atari and MuJoCo games, we verify the explanation fidelity of our method and demonstrate how to employ interpretation to understand agent behavior, discover policy vulnerabilities, remediate policy errors, and even defend against adversarial attacks.

📄 PDF Abstract BibTeX

Code (1)

henrygwb/edge 공식 구현 pytorch

Tasks

Deep Reinforcement LearningMuJoCoMuJoCo Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Variational Inference

Methods 이 논문이 사용한 방법론

Variational Inference 설명 없음

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Interpreting Reinforcement Policies through Local Behaviors

2021-09-29 · Ronny Luss, Amit Dhurandhar, Miao Liu

Many works in explainable AI have focused on explaining black-box classification models. Explaining deep reinforcement learning (RL) policies in a manner that could be understood by domain users has received much less at…

Deep Reinforcement Learning Reinforcement Learning (RL)

CDT: Cascading Decision Trees for Explainable Reinforcement Learning

2020-11-15 · Zihan Ding, Pablo Hernandez-Leal, Gavin Weiguang Ding, Changjian Li 외

Deep Reinforcement Learning (DRL) has recently achieved significant advances in various domains. However, explaining the policy of RL agents still remains an open problem due to several factors, one being the complexity …

Deep Reinforcement Learning Explainable Models Imitation Learning reinforcement-learning+3

Local Explanations for Reinforcement Learning

2022-02-08 · Ronny Luss, Amit Dhurandhar, Miao Liu

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Diverse Exploration for Fast and Safe Policy Improvement

2018-02-22 · Andrew Cohen, Lei Yu, Robert Wright

We study an important yet under-addressed problem of quickly and safely improving policies in online reinforcement learning domains. As its solution, we propose a novel exploration strategy - diverse exploration (DE), wh…

Diversity reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

(When) Are Contrastive Explanations of Reinforcement Learning Helpful?

2022-11-14 · Sanjana Narayanan, Isaac Lage, Finale Doshi-Velez

Global explanations of a reinforcement learning (RL) agent's expected behavior can make it safer to deploy. However, such explanations are often difficult to understand because of the complicated nature of many RL polici…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)