Learning impartial policies for sequential counterfactual explanations using Deep Reinforcement Learning

2023-11-01 · E. Panagiotou, E. Ntoutsi

In the field of explainable Artificial Intelligence (XAI), sequential counterfactual (SCF) examples are often used to alter the decision of a trained classifier by implementing a sequence of modifications to the input instance. Although certain test-time algorithms aim to optimize for each new instance individually, recently Reinforcement Learning (RL) methods have been proposed that seek to learn policies for discovering SCFs, thereby enhancing scalability. As is typical in RL, the formulation of the RL problem, including the specification of state space, actions, and rewards, can often be ambiguous. In this work, we identify shortcomings in existing methods that can result in policies with undesired properties, such as a bias towards specific actions. We propose to use the output probabilities of the classifier to create a more informative reward, to mitigate this effect.

📄 PDF Abstract BibTeX arXiv:2311.00523

Code (0)

등록된 구현이 없습니다.

Tasks

counterfactualDeep Reinforcement LearningExplainable artificial intelligenceExplainable Artificial Intelligence (XAI)reinforcement-learning Reinforcement Learning (RL)

Learning "What-if" Explanations for Sequential Decision-Making

2020-07-02 · ICLR 2021 1 · Ioana Bica, Daniel Jarrett, Alihan Hüyük, Mihaela Van Der Schaar

Building interpretable parameterizations of real-world decision-making on the basis of demonstrated behavior -- i.e. trajectories of observations and actions made by an expert maximizing some unknown reward function -- i…

counterfactual Counterfactual Reasoning Decision Making reinforcement-learning+3

Counterfactual Explanation Policies in RL

2023-07-25 · Shripad V. Deshmukh, Srivatsan R, Supriti Vijay, Jayakumar Subramanian 외

As Reinforcement Learning (RL) agents are increasingly employed in diverse decision-making problems using reward preferences, it becomes important to ensure that policies learned by these frameworks in mapping observatio…

counterfactual Counterfactual Explanation Decision Making Reinforcement Learning (RL)

RACCER: Towards Reachable and Certain Counterfactual Explanations for Reinforcement Learning

2023-03-08 · Jasmina Gajcin, Ivana Dusparic

While reinforcement learning (RL) algorithms have been successfully applied to numerous tasks, their reliance on neural networks makes their behavior difficult to understand and trust. Counterfactual explanations are hum…

counterfactual reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Counterfactual Explanations for Continuous Action Reinforcement Learning

2025-05-19 · Shuyang Dong, Shangtong Zhang, Lu Feng

Reinforcement Learning (RL) has shown great promise in domains like healthcare and robotics but often struggles with adoption due to its lack of interpretability. Counterfactual explanations, which address "what if" scen…

counterfactual reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

TalkToAgent: A Human-centric Explanation of Reinforcement Learning Agents with Large Language Models

2025-09-05 · Haechang Kim, Hao Chen, Can Li, Jong Min Lee arxiv

Explainable Reinforcement Learning (XRL) has emerged as a promising approach in improving the transparency of Reinforcement Learning (RL) agents. However, there remains a gap between complex RL policies and domain expert…

Reinforcement Learning