Using Reinforcement Learning to Validate Empirical Game-Theoretic Analysis: A Continuous Double Auction Study

2016-04-22 · Mason Wright

Empirical game-theoretic analysis (EGTA) has recently been applied successfully to analyze the behavior of large numbers of competing traders in a continuous double auction market. Multiagent simulation methods like EGTA are useful for studying complex strategic environments like a stock market, where it is not feasible to solve analytically for the rational behavior of each agent. A weakness of simulation-based methods in strategic settings, however, is that it is typically impossible to prove that the strategy profile assigned to the simulated agents is stable, as in a Nash equilibrium. I propose using reinforcement learning to analyze the regret of supposed Nash-equilibrium strategy profiles found by EGTA. I have developed a new library of reinforcement learning tools, which I have integrated into an extended version of the market simulator from our prior work. I provide evidence for the effectiveness of our library methods, both on a suite of benchmark problems from the literature, and on non-equilibrium strategy profiles in our market environment. Finally, I use our new reinforcement learning tools to provide evidence that the equilibria found by EGTA in our recent continuous double auction study are likely to have only negligible regret, even with respect to an extended strategy space.

📄 PDF Abstract BibTeX arXiv:1604.06710

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games

2026-02-11 · Kazuki Ota, Takayuki Osa, Motoki Omura, Tatsuya Harada arxiv

Two-player games such as board games have long been used as traditional benchmarks for reinforcement learning. This work revisits a policy optimization method with reverse Kullback-Leibler regularization and entropy regu…

Reinforcement Learning

A2C is a special case of PPO

2022-05-18 · Shengyi Huang, Anssi Kanervisto, Antonin Raffin, Weixun Wang 외

Advantage Actor-critic (A2C) and Proximal Policy Optimization (PPO) are popular deep reinforcement learning algorithms used for game AI in recent years. A common understanding is that A2C and PPO are separate algorithms …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Game Theoretic Rating in N-player general-sum games with Equilibria

2022-10-05 · Luke Marris, Marc Lanctot, Ian Gemp, Shayegan Omidshafiei 외

Rating strategies in a game is an important area of research in game theory and artificial intelligence, and can be applied to any real-world competitive or cooperative setting. Traditionally, only transitive dependencie…

Form

Solving Stochastic Games

2009-12-01 · NeurIPS 2009 12 · Liam M. Dermed, Charles L. Isbell

Solving multi-agent reinforcement learning problems has proven difficult because of the lack of tractable algorithms. We provide the first approximation algorithm which solves stochastic games to within $\epsilon$ relat…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Finite-Time Analysis of Q-Value Iteration for General-Sum Stackelberg Games

2026-04-06 · Narim Jeong, Donghwan Lee arxiv

Reinforcement learning has been successful both empirically and theoretically in single-agent settings, but extending these results to multi-agent reinforcement learning in general-sum Markov games remains challenging. T…

Multi-agent Reinforcement Learning