Reinforcement Learning for Mean Field Games, with Applications to Economics

2021-06-25 · Andrea Angiuli, Jean-Pierre Fouque, Mathieu Lauriere

Mean field games (MFG) and mean field control problems (MFC) are frameworks to study Nash equilibria or social optima in games with a continuum of agents. These problems can be used to approximate competitive or cooperative games with a large finite number of agents and have found a broad range of applications, in particular in economics. In recent years, the question of learning in MFG and MFC has garnered interest, both as a way to compute solutions and as a way to model how large populations of learners converge to an equilibrium. Of particular interest is the setting where the agents do not know the model, which leads to the development of reinforcement learning (RL) methods. After reviewing the literature on this topic, we present a two timescale approach with RL for MFG and MFC, which relies on a unified Q-learning algorithm. The main novelty of this method is to simultaneously update an action-value function and a distribution but with different rates, in a model-free fashion. Depending on the ratio of the two learning rates, the algorithm learns either the MFG or the MFC solution. To illustrate this method, we apply it to a mean field problem of accumulated consumption in finite horizon with HARA utility function, and to a trader's optimal liquidation problem.

📄 PDF Abstract BibTeX arXiv:2106.13755

Code (0)

등록된 구현이 없습니다.

Tasks

Q-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Applications of Mean Field Games in Financial Engineering and Economic Theory

2020-12-09 · Rene Carmona

This is an expanded version of the lecture given at the AMS Short Course on Mean Field Games, on January 13, 2020 in Denver CO. The assignment was to discuss applications of Mean Field Games in finance and economics. I n…

Deep Learning for Mean Field Games and Mean Field Control with Applications to Finance

2021-07-09 · René Carmona, Mathieu Laurière

Financial markets and more generally macro-economic models involve a large number of individuals interacting through variables such as prices resulting from the aggregate behavior of all the agents. Mean field games have…

Provable Fictitious Play for General Mean-Field Games

2020-10-08 · Qiaomin Xie, Zhuoran Yang, Zhaoran Wang, Andreea Minca

We propose a reinforcement learning algorithm for stationary mean-field games, where the goal is to learn a pair of mean-field state and stationary policy that constitutes the Nash equilibrium. When viewing the mean-fiel…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Games

2024-05-01 · Anran Hu, Junzi Zhang

Reinforcement learning for multi-agent games has attracted lots of attention recently. However, given the challenge of solving Nash equilibria for large population games, existing works with guaranteed polynomial complex…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Regularization of the policy updates for stabilizing Mean Field Games

2023-04-04 · Talal Algumaei, Ruben Solozabal, REDA ALAMI, Hakim Hacid 외

This work studies non-cooperative Multi-Agent Reinforcement Learning (MARL) where multiple agents interact in the same environment and whose goal is to maximize the individual returns. Challenges arise when scaling up th…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning