Concept Learning for Interpretable Multi-Agent Reinforcement Learning

2023-02-23 · Renos Zabounidis, Joseph Campbell, Simon Stepputtis, Dana Hughes, Katia Sycara

Multi-agent robotic systems are increasingly operating in real-world environments in close proximity to humans, yet are largely controlled by policy models with inscrutable deep neural network representations. We introduce a method for incorporating interpretable concepts from a domain expert into models trained through multi-agent reinforcement learning, by requiring the model to first predict such concepts then utilize them for decision making. This allows an expert to both reason about the resulting concept policy models in terms of these high-level concepts at run-time, as well as intervene and correct mispredictions to improve performance. We show that this yields improved interpretability and training stability, with benefits to policy performance and sample efficiency in a simulated and real-world cooperative-competitive multi-agent game.

📄 PDF Abstract BibTeX arXiv:2302.12232

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Concept Learning for Cooperative Multi-Agent Reinforcement Learning

2025-07-27 · Zhonghan Ge, Yuanyang Zhu, Chunlin Chen arxiv

Despite substantial progress in applying neural networks (NN) to multi-agent reinforcement learning (MARL) areas, they still largely suffer from a lack of transparency and interoperability. However, its implicit cooperat…

Multi-agent Reinforcement Learning Starcraft II

Prism: Policy Reuse via Interpretable Strategy Mapping in Reinforcement Learning

2026-03-04 · Thomas Pravetz arxiv

We present PRISM (Policy Reuse via Interpretable Strategy Mapping), a framework that grounds reinforcement learning agents' decisions in discrete, causally validated concepts and uses those concepts as a zero-shot transf…

Reinforcement Learning

Learning of Generalizable and Interpretable Knowledge in Grid-Based Reinforcement Learning Environments

2023-09-07 · Manuel Eberhardinger, Johannes Maucher, Setareh Maghsudi

Understanding the interactions of agents trained with deep reinforcement learning is crucial for deploying agents in games or the real world. In the former, unreasonable actions confuse players. In the latter, that effec…

Atari Games Decision Making Deep Reinforcement Learning Language Modeling+4

Interpretable end-to-end Neurosymbolic Reinforcement Learning agents

2024-10-18 · Nils Grandien, Quentin Delfosse, Kristian Kersting

Deep reinforcement learning (RL) agents rely on shortcut learning, preventing them from generalizing to slightly different environments. To address this problem, symbolic method, that use object-centric states, have been…

Atari Games Deep Reinforcement Learning Object reinforcement-learning+2

Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents

2024-01-11 · Quentin Delfosse, Sebastian Sztwiertnia, Mark Rothermel, Wolfgang Stammer 외

Goal misalignment, reward sparsity and difficult credit assignment are only a few of the many issues that make it difficult for deep reinforcement learning (RL) agents to learn optimal policies. Unfortunately, the black-…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)