A Unified Framework for Factorizing Distributional Value Functions for Multi-Agent Reinforcement Learning

2023-06-04 · Wei-Fang Sun, Cheng-Kuang Lee, Simon See, Chun-Yi Lee

In fully cooperative multi-agent reinforcement learning (MARL) settings, environments are highly stochastic due to the partial observability of each agent and the continuously changing policies of other agents. To address the above issues, we proposed a unified framework, called DFAC, for integrating distributional RL with value function factorization methods. This framework generalizes expected value function factorization methods to enable the factorization of return distributions. To validate DFAC, we first demonstrate its ability to factorize the value functions of a simple matrix game with stochastic rewards. Then, we perform experiments on all Super Hard maps of the StarCraft Multi-Agent Challenge and six self-designed Ultra Hard maps, showing that DFAC is able to outperform a number of baselines.

📄 PDF Abstract BibTeX arXiv:2306.02430

Code (1)

j3soon/dfac-extended 공식 구현 pytorch

Tasks

Multi-agent Reinforcement Learning reinforcement-learning SMACStarcraft

DFAC Framework: Factorizing the Value Function via Quantile Mixture for Multi-Agent Distributional Q-Learning

2021-02-16 · Wei-Fang Sun, Cheng-Kuang Lee, Chun-Yi Lee

In fully cooperative multi-agent reinforcement learning (MARL) settings, the environments are highly stochastic due to the partial observability of each agent and the continuously changing policies of the other agents. T…

Multi-agent Reinforcement Learning Q-Learning SMAC SMAC++1

Residual Q-Networks for Value Function Factorizing in Multi-Agent Reinforcement Learning

2022-05-30 · Rafael Pina, Varuna De Silva, Joosep Hook, Ahmet Kondoz

Multi-Agent Reinforcement Learning (MARL) is useful in many problems that require the cooperation and coordination of multiple agents. Learning optimal policies using reinforcement learning in a multi-agent setting can b…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Distributional Reinforcement Learning for Multi-Dimensional Reward Functions

2021-10-26 · NeurIPS 2021 12 · Pushi Zhang, Xiaoyu Chen, Li Zhao, Wei Xiong 외

A growing trend for value-based reinforcement learning (RL) algorithms is to capture more information than scalar value functions in the value network. One of the most well-known methods in this branch is distributional …

Distributional Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Distributional Analysis of Sampling-Based Reinforcement Learning Algorithms

2020-03-27 · Philip Amortila, Doina Precup, Prakash Panangaden, Marc G. Bellemare

We present a distributional approach to theoretical analyses of reinforcement learning algorithms for constant step-sizes. We demonstrate its effectiveness by presenting simple and unified proofs of convergence for a var…

Q-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Unified Wasserstein Distributional Robustness Framework for Adversarial Training

2022-02-27 · ICLR 2022 4 · Tuan Anh Bui, Trung Le, Quan Tran, He Zhao 외

It is well-known that deep neural networks (DNNs) are susceptible to adversarial attacks, exposing a severe fragility of deep learning systems. As the result, adversarial training (AT) method, by incorporating adversaria…