Reducing Overestimation Bias in Multi-Agent Domains Using Double Centralized Critics

2019-10-03 · Johannes Ackermann, Volker Gabler, Takayuki Osa, Masashi Sugiyama

Many real world tasks require multiple agents to work together. Multi-agent reinforcement learning (RL) methods have been proposed in recent years to solve these tasks, but current methods often fail to efficiently learn policies. We thus investigate the presence of a common weakness in single-agent RL, namely value function overestimation bias, in the multi-agent setting. Based on our findings, we propose an approach that reduces this bias by using double centralized critics. We evaluate it on six mixed cooperative-competitive tasks, showing a significant advantage over current methods. Finally, we investigate the application of multi-agent methods to high-dimensional robotic tasks and show that our approach can be used to learn decentralized policies in this domain.

📄 PDF Abstract BibTeX arXiv:1910.01465

Code (3)

JohannesAck/MATD3implementation 공식 구현 tf

JohannesAck/tf2multiagentrl tf

fdcl-gwu/gym-rotor pytorch

Tasks

Multi-agent Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

Dual Ensembled Multiagent Q-Learning with Hypernet Regularizer

2025-02-04 · Yaodong Yang, Guangyong Chen, Hongyao Tang, Furui Liu 외

Overestimation in single-agent reinforcement learning has been extensively studied. In contrast, overestimation in the multiagent setting has received comparatively little attention although it increases with the number …

Q-Learning SMAC SMAC+

Elastic Step DQN: A novel multi-step algorithm to alleviate overestimation in Deep QNetworks

2022-10-07 · Adrian Ly, Richard Dazeley, Peter Vamplew, Francisco Cruz 외

Deep Q-Networks algorithm (DQN) was the first reinforcement learning algorithm using deep neural network to successfully surpass human level performance in a number of Atari learning environments. However, divergent and …

OpenAI Gym

Cross Learning in Deep Q-Networks

2020-09-29 · Xing Wang, Alexander Vinel

In this work, we propose a novel cross Q-learning algorithm, aim at alleviating the well-known overestimation problem in value-based reinforcement learning methods, particularly in the deep Q-networks where the overestim…

Q-Learning reinforcement-learning Reinforcement Learning (RL)

Regularized Softmax Deep Multi-Agent Q-Learning

2021-12-01 · NeurIPS 2021 12 · Ling Pan, Tabish Rashid, Bei Peng, Longbo Huang 외

Tackling overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning, but has received comparatively little attention in the multi-agent setting. In thi…

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+3

Regularized Softmax Deep Multi-Agent $Q$-Learning

2021-03-22 · Ling Pan, Tabish Rashid, Bei Peng, Longbo Huang 외