Computational Performance of Deep Reinforcement Learning to find Nash Equilibria

2021-04-26 · Christoph Graf, Viktor Zobernig, Johannes Schmidt, Claude Klöckl

We test the performance of deep deterministic policy gradient (DDPG), a deep reinforcement learning algorithm, able to handle continuous state and action spaces, to learn Nash equilibria in a setting where firms compete in prices. These algorithms are typically considered model-free because they do not require transition probability functions (as in e.g., Markov games) or predefined functional forms. Despite being model-free, a large set of parameters are utilized in various steps of the algorithm. These are e.g., learning rates, memory buffers, state-space dimensioning, normalizations, or noise decay rates and the purpose of this work is to systematically test the effect of these parameter configurations on convergence to the analytically derived Bertrand equilibrium. We find parameter choices that can reach convergence rates of up to 99%. The reliable convergence may make the method a useful tool to study strategic behavior of firms even in more complex settings. Keywords: Bertrand Equilibrium, Competition in Uniform Price Auctions, Deep Deterministic Policy Gradient Algorithm, Parameter Sensitivity Analysis

📄 PDF Abstract BibTeX arXiv:2104.12895

Code (1)

ckrk/bidding_learning 공식 구현 pytorch

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Smooth Nash Equilibria: Algorithms and Complexity

2023-09-21 · Constantinos Daskalakis, Noah Golowich, Nika Haghtalab, Abhishek Shetty

A fundamental shortcoming of the concept of Nash equilibrium is its computational intractability: approximating Nash equilibria in normal-form games is PPAD-hard. In this paper, inspired by the ideas of smoothed analysis…

Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibria

2022-10-28 · Dong-Ki Kim, Matthew Riemer, Miao Liu, Jakob N. Foerster 외

Multiagent learning settings are inherently more difficult than single-agent learning because each agent interacts with other simultaneously learning agents in a shared environment. An effective approach in multiagent re…

MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Games

2024-05-01 · Anran Hu, Junzi Zhang

Reinforcement learning for multi-agent games has attracted lots of attention recently. However, given the challenge of solving Nash equilibria for large population games, existing works with guaranteed polynomial complex…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning

Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Games

2020-06-15 · NeurIPS 2020 12 · Stephen McAleer, John Lanier, Roy Fox, Pierre Baldi

Finding approximate Nash equilibria in zero-sum imperfect-information games is challenging when the number of information states is large. Policy Space Response Oracles (PSRO) is a deep reinforcement learning algorithm g…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

On Finding Local Nash Equilibria (and Only Local Nash Equilibria) in Zero-Sum Games

2019-01-03 · Eric V. Mazumdar, Michael. I. Jordan, S. Shankar Sastry

We propose local symplectic surgery, a two-timescale procedure for finding local Nash equilibria in two-player zero-sum games. We first show that previous gradient-based algorithms cannot guarantee convergence to local N…

Generative Adversarial Network