Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution Communication

2020-04-01 · Jianyu Su, Stephen Adams, Peter A. Beling

We consider a fully cooperative multi-agent system where agents cooperate to maximize a system's utility in a partial-observable environment. We propose that multi-agent systems must have the ability to (1) communicate and understand the inter-plays between agents and (2) correctly distribute rewards based on an individual agent's contribution. In contrast, most work in this setting considers only one of the above abilities. In this study, we develop an architecture that allows for communication among agents and tailors the system's reward for each individual agent. Our architecture represents agent communication through graph convolution and applies an existing credit assignment structure, counterfactual multi-agent policy gradient (COMA), to assist agents to learn communication by back-propagation. The flexibility of the graph structure enables our method to be applicable to a variety of multi-agent systems, e.g. dynamic systems that consist of varying numbers of agents and static systems with a fixed number of agents. We evaluate our method on a range of tasks, demonstrating the advantage of marrying communication with credit assignment. In the experiments, our proposed method yields better performance than the state-of-art methods, including COMA. Moreover, we show that the communication strategies offers us insights and interpretability of the system's cooperative policies.

📄 PDF Abstract BibTeX arXiv:2004.00470

Code (0)

등록된 구현이 없습니다.

Tasks

counterfactualMulti-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Interpretability 설명 없음

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Competitive Multi-Agent Deep Reinforcement Learning with Counterfactual Thinking

2019-08-13 · Yue Wang, Yao Wan, Chenwei Zhang, Lixin Cui 외

Counterfactual thinking describes a psychological phenomenon that people re-infer the possible results with different solutions about things that have already happened. It helps people to gain more experience from mistak…

counterfactual Decision Making Deep Reinforcement Learning Multi-agent Reinforcement Learning+3

PAC: Assisted Value Factorisation with Counterfactual Predictions in Multi-Agent Reinforcement Learning

2022-06-22 · Hanhan Zhou, Tian Lan, Vaneet Aggarwal

Multi-agent reinforcement learning (MARL) has witnessed significant progress with the development of value function factorization methods. It allows optimizing a joint action-value function through the maximization of fa…

counterfactual Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+4

Graph Convolutional Reinforcement Learning

2018-10-22 · ICLR 2020 1 · Jiechuan Jiang, Chen Dun, Tiejun Huang, Zongqing Lu

Learning to cooperate is crucially important in multi-agent environments. The key is to understand the mutual interplay between agents. However, multi-agent environments are highly dynamic, where agents keep moving and t…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Counterfactual Graph for Multi-Agent LLM Calibration

2026-05-28 · Jiatan Huang, Mingchen Li, Ziming Li, Sunjae Kwon 외 arxiv

Multi-agent LLM systems often treat agreement as evidence: when many agents in a panel give the same answer, that answer is assumed to be more reliable. We show that this assumption can fail after agents communicate. Com…

Safety-Contract Graph Multi-Agent Reinforcement Learning for Autonomous Network Security Response

2026-06-11 · Jose Luis Lima de Jesus Silva arxiv

Autonomous network-security response systems promise to reduce Security Operations Centre (SOC) reaction latency, but reward-only multi-agent reinforcement learning (MARL) can improve security reward while remaining non-…

Multi-agent Reinforcement Learning