Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning

2020-06-19 · Sheng Li, Jayesh K. Gupta, Peter Morales, Ross Allen, Mykel J. Kochenderfer

Multi-agent reinforcement learning (MARL) requires coordination to efficiently solve certain tasks. Fully centralized control is often infeasible in such domains due to the size of joint action spaces. Coordination graph based formalization allows reasoning about the joint action based on the structure of interactions. However, they often require domain expertise in their design. This paper introduces the deep implicit coordination graph (DICG) architecture for such scenarios. DICG consists of a module for inferring the dynamic coordination graph structure which is then used by a graph neural network based module to learn to implicitly reason about the joint actions or values. DICG allows learning the tradeoff between full centralization and decentralization via standard actor-critic methods to significantly improve coordination for domains with large number of agents. We apply DICG to both centralized-training-centralized-execution and centralized-training-decentralized-execution regimes. We demonstrate that DICG solves the relative overgeneralization pathology in predatory-prey tasks as well as outperforms various MARL baselines on the challenging StarCraft II Multi-agent Challenge (SMAC) and traffic junction environments.

📄 PDF Abstract BibTeX arXiv:2006.11438

Code (1)

sisl/DICG 공식 구현 pytorch

Tasks

Graph Neural Network Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)SMAC SMAC+StarcraftStarcraft II

Methods 이 논문이 사용한 방법론

Graph Neural Network 설명 없음

Deep Meta Coordination Graphs for Multi-agent Reinforcement Learning

2025-02-06 · Nikunj Gupta, James Zachary Hare, Rajgopal Kannan, Viktor Prasanna

This paper presents deep meta coordination graphs (DMCG) for learning cooperative policies in multi-agent reinforcement learning (MARL). Coordination graph formulations encode local interactions and accordingly factorize…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Learning Implicit Causal World Models from Multi-Agent Demonstrations

2026-07-28 · Jasorsi Ghosh arxiv

In model-based reinforcement learning, world models exist as internal simulators, but their training often conflates statistical correlations with causal mechanisms. This problem is exacerbated in multi-agent systems whe…

Reinforcement Learning

Coordination Graphs for Constrained Multi-Agent Reinforcement Learning

2026-06-01 · Santiago Amaya-Corredor, Miguel Calvo-Fullana, Anders Jonsson arxiv

Constrained Multi-agent reinforcement learning (CMARL) faces two intertwined challenges: the joint action space grows exponentially with the number of agents, and additional requirements couple agents in ways that reward…

Multi-agent Reinforcement Learning

Distributed Cooperative Multi-Agent Reinforcement Learning with Directed Coordination Graph

2022-01-10 · Gangshan Jing, He Bai, Jemin George, Aranya Chakrabortty 외

Existing distributed cooperative multi-agent reinforcement learning (MARL) frameworks usually assume undirected coordination graphs and communication graphs while estimating a global reward via consensus algorithms for p…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Resolving Implicit Coordination in Multi-Agent Deep Reinforcement Learning with Deep Q-Networks & Game Theory

2020-12-08 · Griffin Adams, Sarguna Janani Padmanabhan, Shivang Shekhar

We address two major challenges of implicit coordination in multi-agent deep reinforcement learning: non-stationarity and exponential growth of state-action space, by combining Deep-Q Networks for policy learning with Na…

Deep Reinforcement Learning OpenAI Gym Reinforcement Learning (RL)