Deep Coordination Graphs
This paper introduces the deep coordination graph (DCG) for collaborative multi-agent reinforcement learning. DCG strikes a flexible trade-off between representational capacity and generalization by factoring the joint value function of all agents according to a coordination graph into payoffs between pairs of agents. The value can be maximized by local message passing along the graph, which allows training of the value function end-to-end with Q-learning. Payoff functions are approximated with deep neural networks that employ parameter sharing and low-rank approximations to significantly improve sample efficiency. We show that DCG can solve predator-prey tasks that highlight the relative overgeneralization pathology, as well as challenging StarCraft II micromanagement tasks.
Code (2)
Tasks
Multi-agent Reinforcement LearningQ-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)StarcraftStarcraft IISimilar Papers 제목 키워드 기반
Deep Meta Coordination Graphs for Multi-agent Reinforcement Learning
This paper presents deep meta coordination graphs (DMCG) for learning cooperative policies in multi-agent reinforcement learning (MARL). Coordination graph formulations encode local interactions and accordingly factorize…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningContext-Aware Sparse Deep Coordination Graphs
Learning sparse coordination graphs adaptive to the coordination dynamics among agents is a long-standing problem in cooperative multi-agent learning. This paper studies this problem and proposes a novel method using the…
graph constructionGraph LearningStarcraftStarcraft IISelf-Organized Polynomial-Time Coordination Graphs
Coordination graph is a promising approach to model agent collaboration in multi-agent reinforcement learning. It conducts a graph-based value factorization and induces explicit coordination among agents to complete comp…
Computational EfficiencyMulti-agent Reinforcement LearningNon-constituent coordination and other coordinative constructions as Dependency Graphs
Time Coordination of Multiple UAVs over Switching Communication Networks with Digraph Topologies
This paper presents a time-coordination algorithm for multiple UAVs executing cooperative missions. Unlike previous algorithms, it does not rely on the assumption that the communication between UAVs is bidirectional. Thu…