홈 › Papers

Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning

2020-03-19 · Tabish Rashid, Mikayel Samvelyan, Christian Schroeder de Witt, Gregory Farquhar, Jakob Foerster, Shimon Whiteson

In many real-world settings, a team of agents must coordinate its behaviour while acting in a decentralised fashion. At the same time, it is often possible to train the agents in a centralised fashion where global state information is available and communication constraints are lifted. Learning joint action-values conditioned on extra state information is an attractive way to exploit centralised learning, but the best strategy for then extracting decentralised policies is unclear. Our solution is QMIX, a novel value-based method that can train decentralised policies in a centralised end-to-end fashion. QMIX employs a mixing network that estimates joint action-values as a monotonic combination of per-agent values. We structurally enforce that the joint-action value is monotonic in the per-agent values, through the use of non-negative weights in the mixing network, which guarantees consistency between the centralised and decentralised policies. To evaluate the performance of QMIX, we propose the StarCraft Multi-Agent Challenge (SMAC) as a new benchmark for deep multi-agent reinforcement learning. We evaluate QMIX on a challenging set of SMAC scenarios and show that it significantly outperforms existing multi-agent reinforcement learning methods.

📄 PDF Abstract BibTeX arXiv:2003.08839

Code (1)

oxwhirl/pymarl 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)SMAC SMAC+Starcraft

QR-MIX: Distributional Value Function Factorisation for Cooperative Multi-Agent Reinforcement Learning

2020-09-09 · Jian Hu, Seth Austin Harding, Haibin Wu, Siyue Hu 외

In Cooperative Multi-Agent Reinforcement Learning (MARL) and under the setting of Centralized Training with Decentralized Execution (CTDE), agents observe and interact with their environment locally and independently. Wi…

Multi-agent Reinforcement Learning quantile regression reinforcement-learning Reinforcement Learning (RL)+3

QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning

2018-03-30 · ICML 2018 7 · Tabish Rashid, Mikayel Samvelyan, Christian Schroeder de Witt, Gregory Farquhar 외

In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or lab…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+3

FACMAC: Factored Multi-Agent Centralised Policy Gradients

2020-03-14 · NeurIPS 2021 12 · Bei Peng, Tabish Rashid, Christian A. Schroeder de Witt, Pierre-Alexandre Kamienny 외

We propose FACtored Multi-Agent Centralised policy gradients (FACMAC), a new method for cooperative multi-agent reinforcement learning in both discrete and continuous action spaces. Like MADDPG, a popular multi-agent act…

MuJoCo Multi-agent Reinforcement Learning Q-Learning Reinforcement Learning+3

QGNN: Value Function Factorisation with Graph Neural Networks

2022-05-25 · Ryan Kortvelesy, Amanda Prorok

In multi-agent reinforcement learning, the use of a global objective is a powerful tool for incentivising cooperation. Unfortunately, it is not sample-efficient to train individual agents with a global reward, because it…

Graph Neural Network Multi-agent Reinforcement Learning Starcraft

PAC: Assisted Value Factorisation with Counterfactual Predictions in Multi-Agent Reinforcement Learning

2022-06-22 · Hanhan Zhou, Tian Lan, Vaneet Aggarwal

Multi-agent reinforcement learning (MARL) has witnessed significant progress with the development of value function factorization methods. It allows optimizing a joint action-value function through the maximization of fa…

counterfactual Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+4