K-level Reasoning for Zero-Shot Coordination in Hanabi

2022-07-14 · NeurIPS 2021 12 · Brandon Cui, Hengyuan Hu, Luis Pineda, Jakob N. Foerster

The standard problem setting in cooperative multi-agent settings is self-play (SP), where the goal is to train a team of agents that works well together. However, optimal SP policies commonly contain arbitrary conventions ("handshakes") and are not compatible with other, independently trained agents or humans. This latter desiderata was recently formalized by Hu et al. 2020 as the zero-shot coordination (ZSC) setting and partially addressed with their Other-Play (OP) algorithm, which showed improved ZSC and human-AI performance in the card game Hanabi. OP assumes access to the symmetries of the environment and prevents agents from breaking these in a mutually incompatible way during training. However, as the authors point out, discovering symmetries for a given environment is a computationally hard problem. Instead, we show that through a simple adaption of k-level reasoning (KLR) Costa Gomes et al. 2006, synchronously training all levels, we can obtain competitive ZSC and ad-hoc teamplay performance in Hanabi, including when paired with a human-like proxy bot. We also introduce a new method, synchronous-k-level reasoning with a best response (SyKLRBR), which further improves performance on our synchronous KLR by co-training a best response.

📄 PDF Abstract BibTeX arXiv:2207.07166

Code (0)

등록된 구현이 없습니다.

Equivariant Networks for Zero-Shot Coordination

2022-10-21 · Darius Muglich, Christian Schroeder de Witt, Elise van der Pol, Shimon Whiteson 외

Successful coordination in Dec-POMDPs requires agents to adopt robust strategies and interpretable styles of play for their partner. A common failure mode is symmetry breaking, when agents arbitrarily converge on one out…

“Other-Play” for Zero-Shot Coordination

2020-01-01 · ICML 2020 1 · Hengyuan Hu, Alexander Peysakhovich, Adam Lerer, Jakob Foerster

We consider the problem of zero-shot coordination - constructing AI agents that can coordinate with novel partners they have not seen before (e.g.humans). Standard Multi-Agent Reinforcement Learning (MARL) methods typica…

Multi-agent Reinforcement Learning Reinforcement Learning (RL)

Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabi

2023-08-20 · Hadi Nekoei, Xutong Zhao, Janarthanan Rajendran, Miao Liu 외

Cooperative Multi-agent Reinforcement Learning (MARL) algorithms with Zero-Shot Coordination (ZSC) have gained significant attention in recent years. ZSC refers to the ability of agents to coordinate zero-shot (without a…

Game of Hanabi Multi-agent Reinforcement Learning Q-Learning

"Other-Play" for Zero-Shot Coordination

2020-03-06 · Hengyuan Hu, Adam Lerer, Alex Peysakhovich, Jakob Foerster

We consider the problem of zero-shot coordination - constructing AI agents that can coordinate with novel partners they have not seen before (e.g. humans). Standard Multi-Agent Reinforcement Learning (MARL) methods typic…

Multi-agent Reinforcement Learning Reinforcement Learning

Sparks of Cooperative Reasoning: LLMs as Strategic Hanabi Agents

2026-01-26 · Mahesh Ramesh, Kaousheik Jayakumar, Aswinkumar Ramkumar, Pavan Thodima 외 arxiv

Cooperative reasoning under incomplete information remains challenging for both humans and multi-agent systems. The card game Hanabi embodies this challenge, requiring theory-of-mind reasoning and strategic communication…

Mathematical Reasoning