Learning Implicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning

2020-07-06 · NeurIPS 2020 12 · Meng Zhou, Ziyu Liu, Pengwei Sui, Yixuan Li, Yuk Ying Chung

We present a multi-agent actor-critic method that aims to implicitly address the credit assignment problem under fully cooperative settings. Our key motivation is that credit assignment among agents may not require an explicit formulation as long as (1) the policy gradients derived from a centralized critic carry sufficient information for the decentralized agents to maximize their joint action value through optimal cooperation and (2) a sustained level of exploration is enforced throughout training. Under the centralized training with decentralized execution (CTDE) paradigm, we achieve the former by formulating the centralized critic as a hypernetwork such that a latent state representation is integrated into the policy gradients through its multiplicative association with the stochastic policies; to achieve the latter, we derive a simple technique called adaptive entropy regularization where magnitudes of the entropy gradients are dynamically rescaled based on the current policy stochasticity to encourage consistent levels of exploration. Our algorithm, referred to as LICA, is evaluated on several benchmarks including the multi-agent particle environments and a set of challenging StarCraft II micromanagement tasks, and we show that LICA significantly outperforms previous methods.

📄 PDF Abstract BibTeX arXiv:2007.02529

Code (1)

mzho7212/LICA 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Starcraft II

Methods 이 논문이 사용한 방법론

HyperNetwork A HyperNetwork is a network that generates weights for a main network. The behavior of the main network is the same with any usual neural network: it learns to map some raw…

Entropy Regularization 설명 없음

Multi-level Advantage Credit Assignment for Cooperative Multi-Agent Reinforcement Learning

2025-08-09 · Xutong Zhao, Yaqi Xie arxiv

Cooperative multi-agent reinforcement learning (MARL) aims to coordinate multiple agents to achieve a common goal. A key challenge in MARL is credit assignment, which involves assessing each agent's contribution to the s…

Multi-agent Reinforcement Learning

Cooperative Multi-Agent Reinforcement Learning with Sequential Credit Assignment

2021-05-21 · NeurIPS 2021 12 · Yifan Zang, Jinmin He, Kai Li, Lily Cao 외

Centralized training with decentralized execution is a standard paradigm for cooperative multi-agent reinforcement learning (MARL), with credit assignment being a major challenge. In this paper, we propose a cooperative …

counterfactual Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+3

Nucleolus Credit Assignment for Effective Coalitions in Multi-agent Reinforcement Learning

2025-03-01 · Yugu Li, Zehong Cao, Jianglin Qiao, Siyi Hu

In cooperative multi-agent reinforcement learning (MARL), agents typically form a single grand coalition based on credit assignment to tackle a composite task, often resulting in suboptimal performance. This paper propos…

Multi-agent Reinforcement Learning Q-Learning Starcraft

Shapley Counterfactual Credits for Multi-Agent Reinforcement Learning

2021-06-01 · Jiahui Li, Kun Kuang, Baoxiang Wang, Furui Liu 외

Centralized Training with Decentralized Execution (CTDE) has been a popular paradigm in cooperative Multi-Agent Reinforcement Learning (MARL) settings and is widely used in many real applications. One of the major challe…