Mean Field Multi-Agent Reinforcement Learning

2018-02-15 · ICML 2018 7 · Yaodong Yang, Rui Luo, Minne Li, Ming Zhou, Wei-Nan Zhang, Jun Wang

Existing multi-agent reinforcement learning methods are limited typically to a small number of agents. When the agent number increases largely, the learning becomes intractable due to the curse of the dimensionality and the exponential growth of agent interactions. In this paper, we present \emph{Mean Field Reinforcement Learning} where the interactions within the population of agents are approximated by those between a single agent and the average effect from the overall population or neighboring agents; the interplay between the two entities is mutually reinforced: the learning of the individual agent's optimal policy depends on the dynamics of the population, while the dynamics of the population change according to the collective patterns of the individual policies. We develop practical mean field Q-learning and mean field Actor-Critic algorithms and analyze the convergence of the solution to Nash equilibrium. Experiments on Gaussian squeeze, Ising model, and battle games justify the learning effectiveness of our mean field approaches. In addition, we report the first result to solve the Ising model via model-free reinforcement learning methods.

📄 PDF Abstract BibTeX arXiv:1802.05438

Code (3)

mlii/mfrl 공식 구현

baoqianwang/iros22_darl1n tf

esmeralday/MARL

Tasks

Multi-agent Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Multi Type Mean Field Reinforcement Learning

2020-02-06 · Sriram Ganapathi Subramanian, Pascal Poupart, Matthew E. Taylor, Nidhi Hegde

Mean field theory provides an effective way of scaling multiagent reinforcement learning algorithms to environments with many agents that can be abstracted by a virtual mean agent. In this paper, we extend mean field mul…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Type Prediction

Partially Observable Mean Field Multi-Agent Reinforcement Learning Based on Graph-Attention

2023-04-25 · Min Yang, Guanjun Liu, Ziyuan Zhou

Traditional multi-agent reinforcement learning algorithms are difficultly applied in a large-scale multi-agent environment. The introduction of mean field theory has enhanced the scalability of multi-agent reinforcement …

Graph Attention Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Discrete-Time Mean Field Control with Environment States

2021-04-30 · Kai Cui, Anam Tahir, Mark Sinzger, Heinz Koeppl

Multi-agent reinforcement learning methods have shown remarkable potential in solving complex multi-agent problems but mostly lack theoretical guarantees. Recently, mean field control and mean field games have been estab…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+2

Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learning

2020-06-21 · Lingxiao Wang, Zhuoran Yang, Zhaoran Wang

Multi-agent reinforcement learning (MARL) achieves significant empirical successes. However, MARL suffers from the curse of many agents. In this paper, we exploit the symmetry of agents in MARL. In the most generic form,…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Breaking the Curse of Many Agents: Provable Mean Embedding $Q$-Iteration for Mean-Field Reinforcement Learning

2020-01-01 · ICML 2020 1 · Lingxiao Wang, Zhuoran Yang, Zhaoran Wang