홈 › Papers

Scalable Deep Reinforcement Learning Algorithms for Mean Field Games

2022-03-22 · Mathieu Laurière, Sarah Perrin, Sertan Girgin, Paul Muller, Ayush Jain, Theophile Cabannes, Georgios Piliouras, Julien Pérolat, Romuald Élie, Olivier Pietquin, Matthieu Geist

Mean Field Games (MFGs) have been introduced to efficiently approximate games with very large populations of strategic agents. Recently, the question of learning equilibria in MFGs has gained momentum, particularly using model-free reinforcement learning (RL) methods. One limiting factor to further scale up using RL is that existing algorithms to solve MFGs require the mixing of approximated quantities such as strategies or $q$-values. This is far from being trivial in the case of non-linear function approximation that enjoy good generalization properties, e.g. neural networks. We propose two methods to address this shortcoming. The first one learns a mixed strategy from distillation of historical data into a neural network and is applied to the Fictitious Play algorithm. The second one is an online mixing method based on regularization that does not require memorizing historical data or previous estimates. It is used to extend Online Mirror Descent. We demonstrate numerically that these methods efficiently enable the use of Deep RL algorithms to solve various MFGs. In addition, we show that these methods outperform SotA baselines from the literature.

📄 PDF Abstract BibTeX arXiv:2203.11973

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning for Finite Space Mean-Field Type Games

2024-09-25 · Kai Shao, Jiacheng Shen, Chijie An, Mathieu Laurière

Mean field type games (MFTGs) describe Nash equilibria between large coalitions: each coalition consists of a continuum of cooperative agents who maximize the average reward of their coalition while interacting non-coope…

Deep Reinforcement Learning Q-Learning Quantization reinforcement-learning+1

Scalable Offline Reinforcement Learning for Mean Field Games

2024-10-23 · Axel Brunnbauer, Julian Lemmel, Zahra Babaiee, Sophie Neubauer 외

Reinforcement learning algorithms for mean-field games offer a scalable framework for optimizing policies in large populations of interacting agents. Existing methods often depend on online interactions or access to syst…

reinforcement-learning Reinforcement Learning

Efficient and Scalable Deep Reinforcement Learning for Mean Field Control Games

2024-12-28 · Nianli Peng, Yilin Wang

Mean Field Control Games (MFCGs) provide a powerful theoretical framework for analyzing systems of infinitely many interacting agents, blending elements from Mean Field Games (MFGs) and Mean Field Control (MFC). However,…

Deep Reinforcement Learning Reinforcement Learning (RL)

Multi Type Mean Field Reinforcement Learning

2020-02-06 · Sriram Ganapathi Subramanian, Pascal Poupart, Matthew E. Taylor, Nidhi Hegde

Mean field theory provides an effective way of scaling multiagent reinforcement learning algorithms to environments with many agents that can be abstracted by a virtual mean agent. In this paper, we extend mean field mul…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Type Prediction

Learning Mean Field Control on Sparse Graphs

2025-01-28 · Christian Fabian, Kai Cui, Heinz Koeppl

Large agent networks are abundant in applications and nature and pose difficult challenges in the field of multi-agent reinforcement learning (MARL) due to their computational and theoretical complexity. While graphon me…

Multi-agent Reinforcement Learning