Experiential Explanations for Reinforcement Learning

2022-10-10 · Amal Alabdulkarim, Madhuri Singh, Gennie Mansi, Kaely Hall, Upol Ehsan, Mark O. Riedl

Reinforcement learning (RL) systems can be complex and non-interpretable, making it challenging for non-AI experts to understand or intervene in their decisions. This is due in part to the sequential nature of RL in which actions are chosen because of their likelihood of obtaining future rewards. However, RL agents discard the qualitative features of their training, making it difficult to recover user-understandable information for "why" an action is chosen. We propose a technique Experiential Explanations to generate counterfactual explanations by training influence predictors along with the RL policy. Influence predictors are models that learn how different sources of reward affect the agent in different states, thus restoring information about how the policy reflects the environment. Two human evaluation studies revealed that participants presented with Experiential Explanations were better able to correctly guess what an agent would do than those presented with other standard types of explanation. Participants also found that Experiential Explanations are more understandable, satisfying, complete, useful, and accurate. Qualitative analysis provides information on the factors of Experiential Explanations that are most useful and the desired characteristics that participants seek from the explanations.

📄 PDF Abstract BibTeX arXiv:2210.04723

Code (1)

amal994/experiential-explanations-rl 공식 구현 pytorch

Tasks

Chunkingcounterfactualreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Notes to Self: Can LLMs Benefit from Experiential Abstractions?

2026-07-22 · Chang Liu, Xinyu Li, Artur Dubrawski arxiv

Humans distill experience into reusable abstractions, e.g., strategies and cautionary reminders, and apply them to gradually solve problems more effectively. We study whether Large Language Models (LLMs) can similarly be…

Reinforcement Learning Logical Reasoning

LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks

2026-07-20 · Tianzhu Ye, Li Dong, Guanheng Chen, He Zhu 외 hf

Reinforcement learning (RL) on open-ended tasks compresses an LLM's rubric-based evaluation into a scalar reward, discarding rich textual feedback and conflating responses with distinct quality profiles. We propose Exper…

Reinforcement Learning

Interpretable experiential learning based on state history and global feedback

2026-05-01 · Anton Kolonin arxiv

A new interpretable experiential learning model based on state history and global feedback is presented. It is capable of learning a behavioral model represented by a transition graph between sets of states, with transit…

Reinforcement Learning OpenAI Gym

Explaining Conditions for Reinforcement Learning Behaviors from Real and Imagined Data

2020-11-17 · Aastha Acharya, Rebecca Russell, Nisar R. Ahmed

The deployment of reinforcement learning (RL) in the real world comes with challenges in calibrating user trust and expectations. As a step toward developing RL systems that are able to communicate their competencies, we…

reinforcement-learning Reinforcement Learning (RL)

Experiential AI

2019-08-06 · Drew Hemment, Ruth Aylett, Vaishak Belle, Dave Murray-Rust 외

Experiential AI is proposed as a new research agenda in which artists and scientists come together to dispel the mystery of algorithms and make their mechanisms vividly apparent. It addresses the challenge of finding nov…