Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learning

2021-09-22 · Roy Zohar, Shie Mannor, Guy Tennenholtz

Cooperative multi-agent reinforcement learning (MARL) faces significant scalability issues due to state and action spaces that are exponentially large in the number of agents. As environments grow in size, effective credit assignment becomes increasingly harder and often results in infeasible learning times. Still, in many real-world settings, there exist simplified underlying dynamics that can be leveraged for more scalable solutions. In this work, we exploit such locality structures effectively whilst maintaining global cooperation. We propose a novel, value-based multi-agent algorithm called LOMAQ, which incorporates local rewards in the Centralized Training Decentralized Execution paradigm. Additionally, we provide a direct reward decomposition method for finding these local rewards when only a global signal is provided. We test our method empirically, showing it scales well compared to other methods, significantly improving performance and convergence speed.

📄 PDF Abstract BibTeX arXiv:2109.10632

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Test 설명 없음

SVDE: Scalable Value-Decomposition Exploration for Cooperative Multi-Agent Reinforcement Learning

2023-03-16 · Shuhan Qi, Shuhao Zhang, Qiang Wang, Jiajia Zhang 외

Value-decomposition methods, which reduce the difficulty of a multi-agent system by decomposing the joint state-action space into local observation-action spaces, have become popular in cooperative multi-agent reinforcem…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Starcraft+1

Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative MARL

2023-09-21 · NeurIPS 2023 11

Value decomposition methods have gained popularity in the field of cooperative multi-agent reinforcement learning. However, almost all existing methods follow the principle of Individual Global Max (IGM) or its variants,…

Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative Multi-Agent Reinforcement Learning

2023-02-04 · Zhiwei Xu, Bin Zhang, Dapeng Li, Guangchong Zhou 외

Multi-agent Reinforcement Learning

Transformer-based Value Function Decomposition for Cooperative Multi-agent Reinforcement Learning in StarCraft

2022-08-15 · Muhammad Junaid Khan, Syed Hammad Ahmed, Gita Sukthankar

The StarCraft II Multi-Agent Challenge (SMAC) was created to be a challenging benchmark problem for cooperative multi-agent reinforcement learning (MARL). SMAC focuses exclusively on the problem of StarCraft micromanagem…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)SMAC+3

Adaptive Value Decomposition with Greedy Marginal Contribution Computation for Cooperative Multi-Agent Reinforcement Learning

2023-02-14 · Shanqi Liu, Yujing Hu, Runze Wu, Dong Xing 외

Real-world cooperation often requires intensive coordination among agents simultaneously. This task has been extensively studied within the framework of cooperative multi-agent reinforcement learning (MARL), and value de…