Value-based CTDE Methods in Symmetric Two-team Markov Game: from Cooperation to Team Competition

2022-11-21 · Pascal Leroy, Jonathan Pisane, Damien Ernst

In this paper, we identify the best learning scenario to train a team of agents to compete against multiple possible strategies of opposing teams. We evaluate cooperative value-based methods in a mixed cooperative-competitive environment. We restrict ourselves to the case of a symmetric, partially observable, two-team Markov game. We selected three training methods based on the centralised training and decentralised execution (CTDE) paradigm: QMIX, MAVEN and QVMix. For each method, we considered three learning scenarios differentiated by the variety of team policies encountered during training. For our experiments, we modified the StarCraft Multi-Agent Challenge environment to create competitive environments where both teams could learn and compete simultaneously. Our results suggest that training against multiple evolving strategies achieves the best results when, for scoring their performances, teams are faced with several strategies.

📄 PDF Abstract BibTeX arXiv:2211.11886

Code (2)

paleroy/competpymarl 공식 구현 pytorch

paleroy/competsmac 공식 구현

Tasks

Starcraft

CTDS: Centralized Teacher with Decentralized Student for Multi-Agent Reinforcement Learning

2022-03-16 · Jian Zhao, Xunhan Hu, Mingyu Yang, Wengang Zhou 외

Due to the partial observability and communication constraints in many multi-agent reinforcement learning (MARL) tasks, centralized training with decentralized execution (CTDE) has become one of the most widely used MARL…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Starcraft+1

Inducing Cooperation via Team Regret Minimization based Multi-Agent Deep Reinforcement Learning

2019-11-18 · Runsheng Yu, Zhenyu Shi, Xinrun Wang, Rundong Wang 외

Existing value-factorized based Multi-Agent deep Reinforce-ment Learning (MARL) approaches are well-performing invarious multi-agent cooperative environment under thecen-tralized training and decentralized execution(CTDE…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-Agent Deep Reinforcement Learning Under Constrained Communications

2026-01-22 · Shahil Shaik, Jonathon M. Smereka, Yue Wang arxiv

Centralized training with decentralized execution (CTDE) has been the dominant paradigm in multi-agent reinforcement learning (MARL), but its reliance on global state information during training introduces scalability, r…

Multi-agent Reinforcement Learning

Centralizing State-Values in Dueling Networks for Multi-Robot Reinforcement Learning Mapless Navigation

2021-12-16 · Enrico Marchesini, Alessandro Farinelli

We study the problem of multi-robot mapless navigation in the popular Centralized Training and Decentralized Execution (CTDE) paradigm. This problem is challenging when each robot considers its path without explicitly sh…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

RACA: Relation-Aware Credit Assignment for Ad-Hoc Cooperation in Multi-Agent Deep Reinforcement Learning

2022-06-02 · Hao Chen, Guangkai Yang, Junge Zhang, Qiyue Yin 외

In recent years, reinforcement learning has faced several challenges in the multi-agent domain, such as the credit assignment issue. Value function factorization emerges as a promising way to handle the credit assignment…

Deep Reinforcement Learning Reinforcement Learning (RL)Relation Zero-shot Generalization