Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL

2022-06-01 · Siyi Hu, Chuanlong Xie, Xiaodan Liang, Xiaojun Chang

Cooperative multi-agent reinforcement learning (MARL) is making rapid progress for solving tasks in a grid world and real-world scenarios, in which agents are given different attributes and goals, resulting in different behavior through the whole multi-agent task. In this study, we quantify the agent's behavior difference and build its relationship with the policy performance via {\bf Role Diversity}, a metric to measure the characteristics of MARL tasks. We define role diversity from three perspectives: action-based, trajectory-based, and contribution-based to fully measure a multi-agent task. Through theoretical analysis, we find that the error bound in MARL can be decomposed into three parts that have a strong relation to the role diversity. The decomposed factors can significantly impact policy optimization on three popular directions including parameter sharing, communication mechanism, and credit assignment. The main experimental platforms are based on {\bf Multiagent Particle Environment (MPE)} and {\bf The StarCraft Multi-Agent Challenge (SMAC). Extensive experiments} clearly show that role diversity can serve as a robust measurement for the characteristics of a multi-agent cooperation task and help diagnose whether the policy fits the current multi-agent system for a better policy performance.

📄 PDF Abstract BibTeX arXiv:2207.05683

Code (0)

등록된 구현이 없습니다.

Tasks

DiversityMulti-agent Reinforcement Learning SMAC SMAC+Starcraft

Role Diversity Matters: A Study of Cooperative Training Strategies for Multi-Agent RL

2021-09-29 · Siyi Hu, Chuanlong Xie, Xiaodan Liang, Xiaojun Chang

Diversity Multi-agent Reinforcement Learning SMAC SMAC++2

Measuring Policy Distance for Multi-Agent Reinforcement Learning

2024-01-20 · Tianyi Hu, Zhiqiang Pu, Xiaolin Ai, Tenghai Qiu 외

Diversity plays a crucial role in improving the performance of multi-agent reinforcement learning (MARL). Currently, many diversity-based methods have been developed to overcome the drawbacks of excessive parameter shari…

Diversity Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Role Play: Learning Adaptive Role-Specific Strategies in Multi-Agent Interactions

2024-11-02 · Weifan Long, Wen Wen, Peng Zhai, Lihua Zhang

Zero-shot coordination problem in multi-agent reinforcement learning (MARL), which requires agents to adapt to unseen agents, has attracted increasing attention. Traditional approaches often rely on the Self-Play (SP) fr…

Diversity Multi-agent Reinforcement Learning Role Embedding

Learned Coordination Conventions in Cooperative MARL: Measuring the Translation Gap Between Theory-Informed Roles and Learned Routing

2026-06-28 · Yoosung Hong arxiv

Role-semantic assignments provide priors over how heterogeneous agents may coordinate, but cooperative MARL systems instead settle on conventions through decentralized, non-stationary learning, with no guarantee that the…

Learning Coordinated Preference for Multi-Objective Multi-Agent Reinforcement Learning

2026-06-12 · Pengxin Wang, Lihao Guo, Yi Xie, Bo Liu 외 arxiv

Cooperative multi-objective multi-agent reinforcement learning (MOMARL) models team decision making under multiple, potentially conflicting objectives. In this setting, conflicts arise not only across objectives but also…

Multi-agent Reinforcement Learning Decision Making