paper-with-me

Papers

Computing and Learning Stationary Mean Field Equilibria with Scalar Interactions: Algorithms and Applications

2025-02-02 · Bar Light

Mean field equilibrium (MFE) has emerged as a computationally tractable solution concept for large dynamic games. However, computing MFE remains challenging due to nonlinearities and the absence of contraction properties, limiting its reliability for counterfactual analysis and comparative statics. This paper focuses on MFE in dynamic models where agents interact through a scalar function of the population distribution, referred to as the scalar interaction function. Such models naturally arise in a wide range of applications involving market dynamics and strategic competition. The main contribution of this paper is to introduce iterative algorithms that leverage the scalar interaction structure and are guaranteed to converge to the MFE under mild assumptions. Leveraging this structure, we also establish an MFE existence result for non-compact state spaces and analytical comparative statics. To the best of our knowledge, these are the first algorithms with global convergence guarantees in such settings. Unlike existing approaches, our algorithms do not rely on monotonicity or contraction properties, significantly broadening their applicability. Furthermore, we provide a model-free algorithm that learns the MFE via simulation and reinforcement learning techniques such as Q-learning and policy gradient methods without requiring prior knowledge of payoff or transition functions. We apply our algorithms to classic models of dynamic competition, such as capacity competition, and to competitive models motivated by online marketplaces, including ridesharing and inventory competition, as well as to social learning models. We show how key market parameters influence equilibrium outcomes through reliable comparative statics in these representative models, providing insights into the design of competitive systems.

📄 PDF Abstract BibTeX arXiv:2502.12024

Code (0)

등록된 구현이 없습니다.

Tasks

counterfactualPolicy Gradient MethodsQ-Learning

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Similar Papers 제목 키워드 기반

MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Games

2024-05-01 · Anran Hu, Junzi Zhang

Reinforcement learning for multi-agent games has attracted lots of attention recently. However, given the challenge of solving Nash equilibria for large population games, existing works with guaranteed polynomial complex…

Multi-agent Reinforcement Learningreinforcement-learningReinforcement Learning

Narrow Operator Models of Stellarator Equilibria in Fourier Zernike Basis

2025-10-15 · Timo Thun, Rory Conlin, Dario Panici, Daniel Böckenhoff arxiv

Numerical computation of the ideal Magnetohydrodynamic (MHD) equilibrium magnetic field is at the base of stellarator optimisation and provides the starting point for solving more sophisticated Partial Differential Equat…

Robustness and Approximation of Discrete-time Mean-field Games under Discounted Cost Criterion

2023-10-16 · Uğur Aydın, Naci Saldi

In this paper, we investigate the robustness of stationary mean-field equilibria in the presence of model uncertainties, specifically focusing on infinite-horizon discounted cost functions. To achieve this, we initially …

Quantization

Mean-Field Learning: a Survey

2012-10-17 · Hamidou Tembine, Raul Tempone, Pedro Vilanova

In this paper we study iterative procedures for stationary equilibria in games with large number of players. Most of learning algorithms for games with continuous action spaces are limited to strict contraction best repl…

global-optimizationSurvey

A Black-box Approach for Non-stationary Multi-agent Reinforcement Learning

2023-06-12 · Haozhe Jiang, Qiwen Cui, Zhihan Xiong, Maryam Fazel 외

We investigate learning the equilibria in non-stationary multi-agent systems and address the challenges that differentiate multi-agent learning from single-agent learning. Specifically, we focus on games with bandit feed…

Multi-agent Reinforcement Learningreinforcement-learning