Boolean Decision Rules for Reinforcement Learning Policy Summarisation

2022-07-18 · James McCarthy, Rahul Nair, Elizabeth Daly, Radu Marinescu, Ivana Dusparic

Explainability of Reinforcement Learning (RL) policies remains a challenging research problem, particularly when considering RL in a safety context. Understanding the decisions and intentions of an RL policy offer avenues to incorporate safety into the policy by limiting undesirable actions. We propose the use of a Boolean Decision Rules model to create a post-hoc rule-based summary of an agent's policy. We evaluate our proposed approach using a DQN agent trained on an implementation of a lava gridworld and show that, given a hand-crafted feature representation of this gridworld, simple generalised rules can be created, giving a post-hoc explainable summary of the agent's policy. We discuss possible avenues to introduce safety into a RL agent's policy by using rules generated by this rule-based model as constraints imposed on the agent's policy, as well as discuss how creating simple rule summaries of an agent's policy may help in the debugging process of RL agents.

📄 PDF Abstract BibTeX arXiv:2207.08651

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Towards the Use of Deep Reinforcement Learning with Global Policy For Query-based Extractive Summarisation

2017-11-10 · Diego Molla

Supervised approaches for text summarisation suffer from the problem of mismatch between the target labels/scores of individual sentences and the evaluation score of the final summary. Reinforcement learning can solve th…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Reward Learning for Efficient Reinforcement Learning in Extractive Document Summarisation

2019-07-30 · Yang Gao, Christian M. Meyer, Mohsen Mesgar, Iryna Gurevych

Document summarisation can be formulated as a sequential decision-making problem, which can be solved by Reinforcement Learning (RL) algorithms. The predominant RL paradigm for summarisation learns a cross-input policy, …

Decision Making Learning-To-Rank reinforcement-learning Reinforcement Learning+2

Towards the Use of Deep Reinforcement Learning with Global Policy for Query-based Extractive Summarisation

2017-12-01 · ALTA 2017 12 · Diego Moll{\'a}-Aliod

Deep Reinforcement Learning Reinforcement Learning

Video Summarisation by Classification with Deep Reinforcement Learning

2018-07-09 · Kaiyang Zhou, Tao Xiang, Andrea Cavallaro

Most existing video summarisation methods are based on either supervised or unsupervised learning. In this paper, we propose a reinforcement learning-based weakly supervised method that exploits easy-to-obtain, video-lev…

Classification Decision Making Deep Reinforcement Learning General Classification+5

On the Computational Intelligibility of Boolean Classifiers

2021-04-13 · Gilles Audemard, Steve Bellart, Louenas Bounia, Frédéric Koriche 외

In this paper, we investigate the computational intelligibility of Boolean classifiers, characterized by their ability to answer XAI queries in polynomial time. The classifiers under consideration are decision trees, DNF…

Explainable Artificial Intelligence (XAI)