Value-Decomposition Multi-Agent Actor-Critics

2020-07-24 · Jianyu Su, Stephen Adams, Peter A. Beling

The exploitation of extra state information has been an active research area in multi-agent reinforcement learning (MARL). QMIX represents the joint action-value using a non-negative function approximator and achieves the best performance, by far, on multi-agent benchmarks, StarCraft II micromanagement tasks. However, our experiments show that, in some cases, QMIX is incompatible with A2C, a training paradigm that promotes algorithm training efficiency. To obtain a reasonable trade-off between training efficiency and algorithm performance, we extend value-decomposition to actor-critics that are compatible with A2C and propose a novel actor-critic framework, value-decomposition actor-critics (VDACs). We evaluate VDACs on the testbed of StarCraft II micromanagement tasks and demonstrate that the proposed framework improves median performance over other actor-critic methods. Furthermore, we use a set of ablation experiments to identify the key factors that contribute to the performance of VDACs.

📄 PDF Abstract BibTeX arXiv:2007.12306

Code (1)

hahayonghuming/VDACs 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning Reinforcement Learning (RL)StarcraftStarcraft II

Methods 이 논문이 사용한 방법론

A2C A2C, or Advantage Actor Critic, is a synchronous version of the A3C policy gradient method. As an alternative to the asynchronous…

ACPO: Agent-Chained Policy Optimization for Multi-Agent Reinforcement Learning

2026-06-29 · Daiki E. Matsunaga, Junho Na, Tri Wahyu Guntara, Scott Sanner 외 arxiv

Cooperative tasks in Multi-Agent Reinforcement Learning (MARL) require agents to collectively maximize a shared return. Under the Centralized Training with Decentralized Execution (CTDE) paradigm, policy gradients have r…

Multi-agent Reinforcement Learning

Multi-Agent Cross-Entropy Method with Monotonic Nonlinear Critic Decomposition

2025-11-24 · Yan Wang, Ke Deng, Yongli Ren arxiv

Cooperative multi-agent reinforcement learning (MARL) commonly adopts centralized training with decentralized execution (CTDE), where centralized critics leverage global information to guide decentralized actors. However…

Multi-agent Reinforcement Learning

REValueD: Regularised Ensemble Value-Decomposition for Factorisable Markov Decision Processes

2024-01-16 · David Ireland, Giovanni Montana

Discrete-action reinforcement learning algorithms often falter in tasks with high-dimensional discrete action spaces due to the vast number of possible actions. A recent advancement leverages value-decomposition, a conce…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning

Off-Policy Multi-Agent Decomposed Policy Gradients

2020-07-24 · Yihan Wang, Beining Han, Tonghan Wang, Heng Dong 외

Multi-agent policy gradient (MAPG) methods recently witness vigorous progress. However, there is a significant performance discrepancy between MAPG methods and state-of-the-art multi-agent value-based approaches. In this…

Multi-agent Reinforcement Learning Starcraft Starcraft II

DOP: Off-Policy Multi-Agent Decomposed Policy Gradients

2021-01-01 · ICLR 2021 1 · Yihan Wang, Beining Han, Tonghan Wang, Heng Dong 외