Automatic Grouping for Efficient Cooperative Multi-Agent Reinforcement Learning

2023-09-21 · NeurIPS 2023 11

Grouping is ubiquitous in natural systems and is essential for promoting efficiency in team coordination. This paper proposes a novel formulation of Group-oriented Multi-Agent Reinforcement Learning (GoMARL), which learns automatic grouping without domain knowledge for efficient cooperation. In contrast to existing approaches that attempt to directly learn the complex relationship between the joint action-values and individual utilities, we empower grouping as a bridge to model the connection between small sets of agents and encourage cooperation among them, thereby improving the learning efficiency of the whole team. In particular, we factorize the joint action-values as a combination of group-wise values, which guide agents to improve their policies in a fine-grained fashion. We present an automatic grouping mechanism to generate dynamic groups and group action-values. We further introduce a hierarchical control for policy learning that drives the agents in the same group to specialize in similar policies and possess diverse strategies for various groups. Experiments on the StarCraft II micromanagement tasks and Google Research Football scenarios verify our method's effectiveness. Extensive component studies show how grouping works and enhances performance.

📄 PDF Abstract BibTeX

Code (1)

zyfsjycc/gomarl 공식 구현 pytorch

Hierarchical Reinforcement Learning for Optimal Agent Grouping in Cooperative Systems

2025-01-11 · Liyuan Hu

This paper presents a hierarchical reinforcement learning (RL) approach to address the agent grouping or pairing problem in cooperative multi-agent systems. The goal is to simultaneously learn the optimal grouping and ag…

Decision Making Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Learning Bilateral Team Formation in Cooperative Multi-Agent Reinforcement Learning

2025-06-24 · Koorosh Moslemi, Chi-Guhn Lee

Team formation and the dynamics of team-based learning have drawn significant interest in the context of Multi-Agent Reinforcement Learning (MARL). However, existing studies primarily focus on unilateral groupings, prede…

Multi-agent Reinforcement Learning

HARP: Human-Assisted Regrouping with Permutation Invariant Critic for Multi-Agent Reinforcement Learning

2024-09-18 · Huawen Hu, Enze Shi, Chenxi Yue, Shuocun Yang 외

Human-in-the-loop reinforcement learning integrates human expertise to accelerate agent learning and provide critical guidance and feedback in complex fields. However, many existing approaches focus on single-agent tasks…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

QTypeMix: Enhancing Multi-Agent Cooperative Strategies through Heterogeneous and Homogeneous Value Decomposition

2024-08-12 · Songchen Fu, Shaojing Zhao, Ta Li, Yonghong Yan

In multi-agent cooperative tasks, the presence of heterogeneous agents is familiar. Compared to cooperation among homogeneous agents, collaboration requires considering the best-suited sub-tasks for each agent. However, …

Multi-agent Reinforcement Learning SMAC SMAC+

Inducing Cooperative behaviour in Sequential-Social dilemmas through Multi-Agent Reinforcement Learning using Status-Quo Loss

2020-01-15 · Pinkesh Badjatiya, Mausoom Sarkar, Abhishek Sinha, Siddharth Singh 외

In social dilemma situations, individual rationality leads to sub-optimal group outcomes. Several human engagements can be modeled as a sequential (multi-step) social dilemmas. However, in contrast to humans, Deep Reinfo…

Clustering Deep Reinforcement Learning Multi-agent Reinforcement Learning Reinforcement Learning