Explainable Reinforcement Learning via a Causal World Model
Generating explanations for reinforcement learning (RL) is challenging as actions may produce long-term effects on the future. In this paper, we develop a novel framework for explainable RL by learning a causal world model without prior knowledge of the causal structure of the environment. The model captures the influence of actions, allowing us to interpret the long-term effects of actions through causal chains, which present how actions influence environmental variables and finally lead to rewards. Different from most explanatory models which suffer from low accuracy, our model remains accurate while improving explainability, making it applicable in model-based learning. As a result, we demonstrate that our causal model can serve as the bridge between explainability and learning.
Code (1)
Tasks
modelreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Explainable Reinforcement Learning Through a Causal Lens
Prevalent theories in cognitive science propose that humans understand and represent the knowledge of the world through causal relationships. In making sense of the world, we build causal models in our mind to encode cau…
counterfactualreinforcement-learningReinforcement LearningReinforcement Learning (RL)+2Causal versus Marginal Shapley Values for Robotic Lever Manipulation Controlled using Deep Reinforcement Learning
We investigate the effect of including domain knowledge about a robotic system's causal relations when generating explanations. To this end, we compare two methods from explainable artificial intelligence, the popular Ke…
Deep Reinforcement LearningExplainable artificial intelligenceReinforcement Learning (RL)Inherently Explainable Reinforcement Learning in Natural Language
We focus on the task of creating a reinforcement learning agent that is inherently explainable -- with the ability to produce immediate local explanations by thinking out loud while performing a task and analyzing entire…
Graph Attentionreinforcement-learningReinforcement LearningReinforcement Learning (RL)Causality in Neural Networks -- An Extended Abstract
Causal reasoning is the main learning and explanation tool used by humans. AI systems should possess causal reasoning capabilities to be deployed in the real world with trust and reliability. Introducing the ideas of cau…
BIG-bench Machine LearningDisentanglementExplainable ModelsFairness+1ReCCoVER: Detecting Causal Confusion for Explainable Reinforcement Learning
Despite notable results in various fields over the recent years, deep reinforcement learning (DRL) algorithms lack transparency, affecting user trust and hindering their deployment to high-risk tasks. Causal confusion re…
Deep Reinforcement Learningfeature selectionreinforcement-learningReinforcement Learning+1