Policy Space Response Oracles: A Survey

2024-03-04 · Ariyan Bighashdel, Yongzhao Wang, Stephen Mcaleer, Rahul Savani, Frans A. Oliehoek

Game theory provides a mathematical way to study the interaction between multiple decision makers. However, classical game-theoretic analysis is limited in scalability due to the large number of strategies, precluding direct application to more complex scenarios. This survey provides a comprehensive overview of a framework for large games, known as Policy Space Response Oracles (PSRO), which holds promise to improve scalability by focusing attention on sufficient subsets of strategies. We first motivate PSRO and provide historical context. We then focus on the strategy exploration problem for PSRO: the challenge of assembling effective subsets of strategies that still represent the original game well with minimum computational cost. We survey current research directions for enhancing the efficiency of PSRO, and explore the applications of PSRO across various domains. We conclude by discussing open questions and future research.

📄 PDF Abstract BibTeX arXiv:2403.02227

Code (0)

등록된 구현이 없습니다.

Tasks

PositionSurvey

Methods 이 논문이 사용한 방법론

Focus 설명 없음

Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Models

2026-03-10 · Daniel Hennes, Zun Li, John Schultz, Marc Lanctot arxiv

Recent advances in multi-agent reinforcement learning, particularly Policy-Space Response Oracles (PSRO), have enabled the computation of approximate game-theoretic equilibria in increasingly complex domains. However, th…

Multi-agent Reinforcement Learning Code Generation

Fusion-PSRO: Nash Policy Fusion for Policy Space Response Oracles

2024-05-31 · Jiesong Lian, Yucong Huang, Chengdong Ma, Mingzhi Wang 외

For solving zero-sum games involving non-transitivity, a useful approach is to maintain a policy population to approximate the Nash Equilibrium (NE). Previous studies have shown that the Policy Space Response Oracles (PS…

Multi-agent Reinforcement Learning

Global Policy-Space Response Oracles for Two-Player Zero-Sum Games

2026-05-27 · Junyu Zhang, Feihong Yang, Jian Wang, Chao Wang 외 arxiv

The Policy-Space Response Oracles (PSRO) framework scales equilibrium computation to large zero-sum games by iteratively expanding a restricted strategy set using deep reinforcement learning (DRL). A central challenge is…

Reinforcement Learning

Sample-Efficient Policy Space Response Oracles with Joint Experience Best Response

2026-02-06 · Ariyan Bighashdel, Thiago D. Simão, Frans A. Oliehoek arxiv

Multi-agent reinforcement learning (MARL) offers a scalable alternative to exact game-theoretic analysis but suffers from non-stationarity and the need to maintain diverse populations of strategies that capture non-trans…

Multi-agent Reinforcement Learning Offline RL

Simulation-Free PSRO: Removing Game Simulation from Policy Space Response Oracles

2025-12-30 · Yingzhuo Liu, Shuodi Liu, Weijun Luo, Liuyu Xiang 외 arxiv

Policy Space Response Oracles (PSRO) combines game-theoretic equilibrium computation with learning and is effective in approximating Nash Equilibrium in zero-sum games. However, the computational cost of PSRO has become …