Explainable Action Advising for Multi-Agent Reinforcement Learning

2022-11-15 · Yue Guo, Joseph Campbell, Simon Stepputtis, Ruiyu Li, Dana Hughes, Fei Fang, Katia Sycara

Action advising is a knowledge transfer technique for reinforcement learning based on the teacher-student paradigm. An expert teacher provides advice to a student during training in order to improve the student's sample efficiency and policy performance. Such advice is commonly given in the form of state-action pairs. However, it makes it difficult for the student to reason with and apply to novel states. We introduce Explainable Action Advising, in which the teacher provides action advice as well as associated explanations indicating why the action was chosen. This allows the student to self-reflect on what it has learned, enabling advice generalization and leading to improved sample efficiency and learning performance - even in environments where the teacher is sub-optimal. We empirically show that our framework is effective in both single-agent and multi-agent scenarios, yielding improved policy returns and convergence rates when compared to state-of-the-art methods

📄 PDF Abstract BibTeX arXiv:2211.07882

Code (1)

sophieyueguo/explainable_action_advising 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Agent-Aware Training for Agent-Agnostic Action Advising in Deep Reinforcement Learning

2023-11-28 · Yaoquan Wei, Shunyu Liu, Jie Song, Tongya Zheng 외

Action advising endeavors to leverage supplementary guidance from expert teachers to alleviate the issue of sampling inefficiency in Deep Reinforcement Learning (DRL). Previous agent-specific action advising methods are …

Atari Games Deep Reinforcement Learning

A Broad-persistent Advising Approach for Deep Interactive Reinforcement Learning in Robotic Environments

2021-10-15 · Hung Son Nguyen, Francisco Cruz, Richard Dazeley

Deep Reinforcement Learning (DeepRL) methods have been widely used in robotics to learn about the environment and acquire behaviors autonomously. Deep Interactive Reinforcement Learning (DeepIRL) includes interactive fee…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Teaching on a Budget in Multi-Agent Deep Reinforcement Learning

2019-04-19 · Ercüment İlhan, Jeremy Gow, Diego Perez-Liebana

Deep Reinforcement Learning (RL) algorithms can solve complex sequential decision tasks successfully. However, they have a major drawback of having poor sample efficiency which can often be tackled by knowledge reuse. In…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

CCKS: Consensus-based Communication and Knowledge Sharing

2026-06-10 · Jinyuan Zu, Xiaowei Lv, Yongcai Wang, Deying Li 외 arxiv

In Decentralized Training and Decentralized Execution (DTDE) for cooperative Multi-Agent Reinforcement Learning (MARL), action-advising-based knowledge sharing promotes interpretable and scalable cooperation among agents…

Multi-agent Reinforcement Learning Contrastive Learning Starcraft II

Learning Hierarchical Teaching Policies for Cooperative Agents

2019-03-07 · Dong-Ki Kim, Miao Liu, Shayegan Omidshafiei, Sebastian Lopez-Cot 외

Collective learning can be greatly enhanced when agents effectively exchange knowledge with their peers. In particular, recent work studying agents that learn to teach other teammates has demonstrated that action advisin…

Transfer Learning