Toward Interpretable Deep Reinforcement Learning with Linear Model U-Trees

2018-07-16 · Guiliang Liu, Oliver Schulte, Wang Zhu, Qingcan Li

Deep Reinforcement Learning (DRL) has achieved impressive success in many applications. A key component of many DRL models is a neural network representing a Q function, to estimate the expected cumulative reward following a state-action pair. The Q function neural network contains a lot of implicit knowledge about the RL problems, but often remains unexamined and uninterpreted. To our knowledge, this work develops the first mimic learning framework for Q functions in DRL. We introduce Linear Model U-trees (LMUTs) to approximate neural network predictions. An LMUT is learned using a novel on-line algorithm that is well-suited for an active play setting, where the mimic learner observes an ongoing interaction between the neural net and the environment. Empirical evaluation shows that an LMUT mimics a Q function substantially better than five baseline methods. The transparent tree structure of an LMUT facilitates understanding the network's learned knowledge by analyzing feature influence, extracting rules, and highlighting the super-pixels in image inputs.

📄 PDF Abstract BibTeX arXiv:1807.05887

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Interpretable Reinforcement Learning with Ensemble Methods

2018-09-19 · Alexander Brown, Marek Petrik

We propose to use boosted regression trees as a way to compute human-interpretable solutions to reinforcement learning problems. Boosting combines several regression trees to improve their accuracy without significantly …

BIG-bench Machine Learning Interpretable Machine Learning regression reinforcement-learning+2

MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees

2022-09-15 · Zichuan Liu, Yuanyang Zhu, Zhi Wang, Yang Gao 외

While achieving tremendous success in various fields, existing multi-agent reinforcement learning (MARL) with a black-box neural network makes decisions in an opaque manner that hinders humans from understanding the lear…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Starcraft+1

Optimization Methods for Interpretable Differentiable Decision Trees in Reinforcement Learning

2019-03-22 · Andrew Silva, Taylor Killian, Ivan Dario Jimenez Rodriguez, Sung-Hyun Son 외

Decision trees are ubiquitous in machine learning for their ease of use and interpretability. Yet, these models are not typically employed in reinforcement learning as they cannot be updated online via stochastic gradien…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Fast Linear Model Trees by PILOT

2023-02-08 · Jakob Raymaekers, Peter J. Rousseeuw, Tim Verdonck, Ruicong Yao

Linear model trees are regression trees that incorporate linear models in the leaf nodes. This preserves the intuitive interpretation of decision trees and at the same time enables them to better capture linear relations…

model Model Selection regression

CLARITree: Cholesky and Lookahead Accelerations for Regression with Interpretable Piecewise Linear Trees

2026-06-11 · Yixiao Wang, Hayden McTavish, Varun Babbar, Margo Seltzer 외 arxiv

Regression trees are among the most interpretable yet expressive model classes in machine learning. Historically, greedy induction has been the dominant approach for constructing well-performing regression trees. While o…

Computational Efficiency