Programmatically Interpretable Reinforcement Learning

2018-04-06 · ICML 2018 7 · Abhinav Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, Swarat Chaudhuri

We present a reinforcement learning framework, called Programmatically Interpretable Reinforcement Learning (PIRL), that is designed to generate interpretable and verifiable agent policies. Unlike the popular Deep Reinforcement Learning (DRL) paradigm, which represents policies by neural networks, PIRL represents policies using a high-level, domain-specific programming language. Such programmatic policies have the benefits of being more easily interpreted than neural networks, and being amenable to verification by symbolic methods. We propose a new method, called Neurally Directed Program Search (NDPS), for solving the challenging nonsmooth optimization problem of finding a programmatic policy with maximal reward. NDPS works by first learning a neural policy network using DRL, and then performing a local search over programmatic policies that seeks to minimize a distance from this neural "oracle". We evaluate NDPS on the task of learning to drive a simulated car in the TORCS car-racing environment. We demonstrate that NDPS is able to discover human-readable policies that pass some significant performance bars. We also show that PIRL policies can have smoother trajectories, and can be more easily transferred to environments not encountered during training, than corresponding policies discovered by DRL.

📄 PDF Abstract BibTeX arXiv:1804.02477

Code (0)

등록된 구현이 없습니다.

Tasks

Car RacingDeep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Programmatic Reinforcement Learning without Oracles

2021-09-29 · ICLR 2022 4 · Wenjie Qiu, He Zhu

Deep reinforcement learning (RL) has led to encouraging successes in many challenging control tasks. However, a deep RL model lacks interpretability due to the difficulty of identifying how the model's control logic rela…

Bilevel Optimization Deep Reinforcement Learning Policy Gradient Methods reinforcement-learning+2

Synthesizing Programmatic Policies with Actor-Critic Algorithms and ReLU Networks

2023-08-04 · Spyros Orfanos, Levi H. S. Lelis

Programmatically Interpretable Reinforcement Learning (PIRL) encodes policies in human-readable computer programs. Novel algorithms were recently introduced with the goal of handling the lack of gradient signal to guide …

Translation

MDP environments for the OpenAI Gym

2017-09-26 · Andreas Kirsch

The OpenAI Gym provides researchers and enthusiasts with simple to use environments for reinforcement learning. Even the simplest environment have a level of complexity that can obfuscate the inner workings of RL approac…

OpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Beyond Learning on Molecules by Weakly Supervising on Molecules

2026-02-04 · Gordan Prastalo, Kevin Maik Jablonka arxiv

Molecular representations are inherently task-dependent, yet most pre-trained molecular encoders are not. Task conditioning promises representations that reorganize based on task descriptions, but existing approaches rel…

Molecular Property Prediction

On quantitative aspects of model interpretability

2020-07-15 · An-phi Nguyen, María Rodríguez Martínez

Despite the growing body of work in interpretable machine learning, it remains unclear how to evaluate different explainability methods without resorting to qualitative assessment and user-studies. While interpretability…

Interpretable Machine Learning model