MMD-MIX: Value Function Factorisation with Maximum Mean Discrepancy for Cooperative Multi-Agent Reinforcement Learning

2021-06-22 · Zhiwei Xu, Dapeng Li, Yunpeng Bai, Guoliang Fan

In the real world, many tasks require multiple agents to cooperate with each other under the condition of local observations. To solve such problems, many multi-agent reinforcement learning methods based on Centralized Training with Decentralized Execution have been proposed. One representative class of work is value decomposition, which decomposes the global joint Q-value $Q_\text{jt}$ into individual Q-values $Q_a$ to guide individuals' behaviors, e.g. VDN (Value-Decomposition Networks) and QMIX. However, these baselines often ignore the randomness in the situation. We propose MMD-MIX, a method that combines distributional reinforcement learning and value decomposition to alleviate the above weaknesses. Besides, to improve data sampling efficiency, we were inspired by REM (Random Ensemble Mixture) which is a robust RL algorithm to explicitly introduce randomness into the MMD-MIX. The experiments demonstrate that MMD-MIX outperforms prior baselines in the StarCraft Multi-Agent Challenge (SMAC) environment.

📄 PDF Abstract BibTeX arXiv:2106.11652

Code (0)

등록된 구현이 없습니다.

Tasks

Distributional Reinforcement LearningMulti-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)SMAC SMAC+Starcraft

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

REM Random Ensemble Mixture (REM) is an easy to implement extension of DQN inspired by Dropout.…

Utilizing Maximum Mean Discrepancy Barycenter for Propagating the Uncertainty of Value Functions in Reinforcement Learning

2024-03-31 · Srinjoy Roy, Swagatam Das

Accounting for the uncertainty of value functions boosts exploration in Reinforcement Learning (RL). Our work introduces Maximum Mean Discrepancy Q-Learning (MMD-QL) to improve Wasserstein Q-Learning (WQL) for uncertaint…

Atari Games Q-Learning Reinforcement Learning (RL)

Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation

2017-07-13 · Thomas Brouwer, Jes Frellsen, Pietro Lió

In this paper, we study the trade-offs of different inference approaches for Bayesian matrix factorisation methods, which are commonly used for predicting missing values, and for finding patterns in the data. In particul…

Bayesian Inference Missing Values Model Selection

QGNN: Value Function Factorisation with Graph Neural Networks

2022-05-25 · Ryan Kortvelesy, Amanda Prorok

In multi-agent reinforcement learning, the use of a global objective is a powerful tool for incentivising cooperation. Unfortunately, it is not sample-efficient to train individual agents with a global reward, because it…

Graph Neural Network Multi-agent Reinforcement Learning Starcraft

Generative neural networks for characteristic functions

2024-01-09 · Florian Brück

We provide a simulation algorithm to simulate from a (multivariate) characteristic function, which is only accessible in a black-box format. The method is based on a generative neural network, whose loss function exploit…

Ensemble Transport Filter via Optimized Maximum Mean Discrepancy

2024-07-16 · Dengfei Zeng, Lijian Jiang

In this paper, we present a new ensemble-based filter method by reconstructing the analysis step of the particle filter through a transport map, which directly transports prior particles to posterior particles. The trans…