Cooperative Multi-Agent Policy Gradients with Sub-optimal Demonstration

2018-12-05 · Peixi Peng, Junliang Xing

Many reality tasks such as robot coordination can be naturally modelled as multi-agent cooperative system where the rewards are sparse. This paper focuses on learning decentralized policies for such tasks using sub-optimal demonstration. To learn the multi-agent cooperation effectively and tackle the sub-optimality of demonstration, a self-improving learning method is proposed: On the one hand, the centralized state-action values are initialized by the demonstration and updated by the learned decentralized policy to improve the sub-optimality. On the other hand, the Nash Equilibrium are found by the current state-action value and are used as a guide to learn the policy. The proposed method is evaluated on the combat RTS games which requires a high level of multi-agent cooperation. Extensive experimental results on various combat scenarios demonstrate that the proposed method can learn multi-agent cooperation effectively. It significantly outperforms many state-of-the-art demonstration based approaches.

📄 PDF Abstract BibTeX arXiv:1812.01825

Code (0)

등록된 구현이 없습니다.

Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning

2020-07-06 · NeurIPS 2020 12 · Meng Zhou, Ziyu Liu, Pengwei Sui, Yixuan Li 외

We present a multi-agent actor-critic method that aims to implicitly address the credit assignment problem under fully cooperative settings. Our key motivation is that credit assignment among agents may not require an ex…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Centralized Adaptive Sampling for Reliable Co-Training of Independent Multi-Agent Policies

2025-08-01 · Nicholas E. Corrado, Josiah P. Hanna arxiv

Independent on-policy policy gradient algorithms are widely used for multi-agent reinforcement learning (MARL) in cooperative and no-conflict games, but they are known to converge sub-optimally when each agent's individu…

Multi-agent Reinforcement Learning

Hybrid Differential Reward: Combining Temporal Difference and Action Gradients for Efficient Multi-Agent Reinforcement Learning in Cooperative Driving

2025-11-21 · Ye Han, Lijun Zhang, Dejian Meng, Zhuang Zhang arxiv

In multi-vehicle cooperative driving tasks involving high-frequency continuous control, traditional state-based reward functions suffer from the issue of vanishing reward differences. This phenomenon results in a low sig…

Multi-agent Reinforcement Learning Continuous Control

CORA: Coalitional Rational Advantage Decomposition for Multi-Agent Policy Gradients

2025-06-03 · Mengda Ji, Genjiu Xu, Liying Wang

This work focuses on the credit assignment problem in cooperative multi-agent reinforcement learning (MARL). Sharing the global advantage among agents often leads to suboptimal policy updates as it fails to account for t…

Multi-agent Reinforcement Learning

Cooperative Multi-agent RL with Communication Constraints

2026-01-18 · Nuoya Xiong, Aarti Singh arxiv

Cooperative MARL often assumes frequent access to global information in a data buffer, such as team rewards or other agents' actions, which is typically unrealistic in decentralized MARL systems due to high communication…