GANterfactual-RL: Understanding Reinforcement Learning Agents' Strategies through Visual Counterfactual Explanations

2023-02-24 · Tobias Huber, Maximilian Demmler, Silvan Mertes, Matthew L. Olson, Elisabeth André

Counterfactual explanations are a common tool to explain artificial intelligence models. For Reinforcement Learning (RL) agents, they answer "Why not?" or "What if?" questions by illustrating what minimal change to a state is needed such that an agent chooses a different action. Generating counterfactual explanations for RL agents with visual input is especially challenging because of their large state spaces and because their decisions are part of an overarching policy, which includes long-term decision-making. However, research focusing on counterfactual explanations, specifically for RL agents with visual input, is scarce and does not go beyond identifying defective agents. It is unclear whether counterfactual explanations are still helpful for more complex tasks like analyzing the learned strategies of different agents or choosing a fitting agent for a specific task. We propose a novel but simple method to generate counterfactual explanations for RL agents by formulating the problem as a domain transfer problem which allows the use of adversarial learning techniques like StarGAN. Our method is fully model-agnostic and we demonstrate that it outperforms the only previous method in several computational metrics. Furthermore, we show in a user study that our method performs best when analyzing which strategies different agents pursue.

📄 PDF Abstract BibTeX arXiv:2302.12689

Code (2)

hcmlab/ganterfactual-rl 공식 구현 pytorch

amir-samadi/safe-rl pytorch

Tasks

counterfactualDecision Making reinforcement-learning Reinforcement Learning (RL)

On the Effects of Fine-tuning Language Models for Text-Based Reinforcement Learning

2024-04-15 · Mauricio Gruppi, Soham Dan, Keerthiram Murugesan, Subhajit Chaudhury

Text-based reinforcement learning involves an agent interacting with a fictional environment using observed text and admissible actions in natural language to complete a task. Previous works have shown that agents can su…

reinforcement-learning Reinforcement Learning

Optimal foraging strategies can be learned

2023-03-10 · Gorka Muñoz-Gil, Andrea López-Incera, Lukas J. Fiderer, Hans J. Briegel

The foraging behavior of animals is a paradigm of target search in nature. Understanding which foraging strategies are optimal and how animals learn them are central challenges in modeling animal foraging. While the ques…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Enhancing Cooperation through Selective Interaction and Long-term Experiences in Multi-Agent Reinforcement Learning

2024-05-04 · Tianyu Ren, Xiao-jun Zeng

The significance of network structures in promoting group cooperation within social dilemmas has been widely recognized. Prior studies attribute this facilitation to the assortment of strategies driven by spatial interac…

Attribute Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Visualizing and Understanding Atari Agents

2017-10-31 · ICML 2018 7 · Sam Greydanus, Anurag Koul, Jonathan Dodge, Alan Fern

While deep reinforcement learning (deep RL) agents are effective at maximizing rewards, it is often unclear what strategies they use to do so. In this paper, we take a step toward explaining deep RL agents through a case…

Deep Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

Emergent collective intelligence from massive-agent cooperation and competition

2023-01-04 · HanMo Chen, Stone Tao, Jiaxin Chen, Weihan Shen 외

Inspired by organisms evolving through cooperation and competition between different populations on Earth, we study the emergence of artificial collective intelligence through massive-agent reinforcement learning. To thi…