홈 › Papers

Approximate exploitability: Learning a best response in large games

2020-04-20 · Finbarr Timbers, Nolan Bard, Edward Lockhart, Marc Lanctot, Martin Schmid, Neil Burch, Julian Schrittwieser, Thomas Hubert, Michael Bowling

Researchers have demonstrated that neural networks are vulnerable to adversarial examples and subtle environment changes, both of which one can view as a form of distribution shift. To humans, the resulting errors can look like blunders, eroding trust in these agents. In prior games research, agent evaluation often focused on the in-practice game outcomes. While valuable, such evaluation typically fails to evaluate robustness to worst-case outcomes. Prior research in computer poker has examined how to assess such worst-case performance, both exactly and approximately. Unfortunately, exact computation is infeasible with larger domains, and existing approximations rely on poker-specific knowledge. We introduce ISMCTS-BR, a scalable search-based deep reinforcement learning algorithm for learning a best response to an agent, thereby approximating worst-case performance. We demonstrate the technique in several two-player zero-sum games against a variety of agents, including several AlphaZero-based agents.

📄 PDF Abstract BibTeX arXiv:2004.09677

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learning

Methods 이 논문이 사용한 방법론

AlphaZero AlphaZero is a reinforcement learning agent for playing board games such as Go, chess, and shogi.

ApproxED: Approximate exploitability descent via learned best responses

2023-01-20 · Carlos Martin, Tuomas Sandholm

There has been substantial progress on finding game-theoretic equilibria. Most of that work has focused on games with finite, discrete action spaces. However, many games involving space, time, money, and other fine-grain…

Anytime PSRO for Two-Player Zero-Sum Games

2022-01-19 · Stephen Mcaleer, Kevin Wang, John Lanier, Marc Lanctot 외

Policy space response oracles (PSRO) is a multi-agent reinforcement learning algorithm that has achieved state-of-the-art performance in very large two-player zero-sum games. PSRO is based on the tabular double oracle (D…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Global Policy-Space Response Oracles for Two-Player Zero-Sum Games

2026-05-27 · Junyu Zhang, Feihong Yang, Jian Wang, Chao Wang 외 arxiv

The Policy-Space Response Oracles (PSRO) framework scales equilibrium computation to large zero-sum games by iteratively expanding a restricted strategy set using deep reinforcement learning (DRL). A central challenge is…

Reinforcement Learning

XDO: A Double Oracle Algorithm for Extensive-Form Games

2021-03-11 · NeurIPS 2021 12 · Stephen Mcaleer, John Lanier, Kevin Wang, Pierre Baldi 외

Policy Space Response Oracles (PSRO) is a reinforcement learning (RL) algorithm for two-player zero-sum games that has been empirically shown to find approximate Nash equilibria in large games. Although PSRO is guarantee…

Form Reinforcement Learning (RL)

Computing Approximate Equilibria in Sequential Adversarial Games by Exploitability Descent

2019-03-13 · Edward Lockhart, Marc Lanctot, Julien Pérolat, Jean-Baptiste Lespiau 외

In this paper, we present exploitability descent, a new algorithm to compute approximate equilibria in two-player zero-sum extensive-form games with imperfect information, by direct policy optimization against worst-case…

counterfactual