Computing and Learning Stationary Mean Field Equilibria with Scalar Interactions: Algorithms and Applications
Mean field equilibrium (MFE) has emerged as a computationally tractable solution concept for large dynamic games. However, computing MFE remains challenging due to nonlinearities and the absence of contraction properties, limiting its reliability for counterfactual analysis and comparative statics. This paper focuses on MFE in dynamic models where agents interact through a scalar function of the population distribution, referred to as the scalar interaction function. Such models naturally arise in a wide range of applications involving market dynamics and strategic competition. The main contribution of this paper is to introduce iterative algorithms that leverage the scalar interaction structure and are guaranteed to converge to the MFE under mild assumptions. Leveraging this structure, we also establish an MFE existence result for non-compact state spaces and analytical comparative statics. To the best of our knowledge, these are the first algorithms with global convergence guarantees in such settings. Unlike existing approaches, our algorithms do not rely on monotonicity or contraction properties, significantly broadening their applicability. Furthermore, we provide a model-free algorithm that learns the MFE via simulation and reinforcement learning techniques such as Q-learning and policy gradient methods without requiring prior knowledge of payoff or transition functions. We apply our algorithms to classic models of dynamic competition, such as capacity competition, and to competitive models motivated by online marketplaces, including ridesharing and inventory competition, as well as to social learning models. We show how key market parameters influence equilibrium outcomes through reliable comparative statics in these representative models, providing insights into the design of competitive systems.
Code (0)
등록된 구현이 없습니다.
Tasks
counterfactualPolicy Gradient MethodsQ-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Games
Reinforcement learning for multi-agent games has attracted lots of attention recently. However, given the challenge of solving Nash equilibria for large population games, existing works with guaranteed polynomial complex…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningNarrow Operator Models of Stellarator Equilibria in Fourier Zernike Basis
Numerical computation of the ideal Magnetohydrodynamic (MHD) equilibrium magnetic field is at the base of stellarator optimisation and provides the starting point for solving more sophisticated Partial Differential Equat…
Robustness and Approximation of Discrete-time Mean-field Games under Discounted Cost Criterion
In this paper, we investigate the robustness of stationary mean-field equilibria in the presence of model uncertainties, specifically focusing on infinite-horizon discounted cost functions. To achieve this, we initially …
QuantizationMean-Field Learning: a Survey
In this paper we study iterative procedures for stationary equilibria in games with large number of players. Most of learning algorithms for games with continuous action spaces are limited to strict contraction best repl…
global-optimizationSurveyA Black-box Approach for Non-stationary Multi-agent Reinforcement Learning
We investigate learning the equilibria in non-stationary multi-agent systems and address the challenges that differentiate multi-agent learning from single-agent learning. Specifically, we focus on games with bandit feed…
Multi-agent Reinforcement Learningreinforcement-learning