Posterior sampling for multi-agent reinforcement learning: solving extensive games with imperfect information

2020-05-01 · ICLR 2020 1 · Yichi Zhou, Jialian Li, Jun Zhu

Posterior sampling for reinforcement learning (PSRL) is a useful framework for making decisions in an unknown environment. PSRL maintains a posterior distribution of the environment and then makes planning on the environment sampled from the posterior distribution. Though PSRL works well on single-agent reinforcement learning problems, how to apply PSRL to multi-agent reinforcement learning problems is relatively unexplored. In this work, we extend PSRL to two-player zero-sum extensive-games with imperfect information (TEGI), which is a class of multi-agent systems. More specifically, we combine PSRL with counterfactual regret minimization (CFR), which is the leading algorithm for TEGI with a known environment. Our main contribution is a novel design of interaction strategies. With our interaction strategies, our algorithm provably converges to the Nash Equilibrium at a rate of $O(\sqrt{\log T/T})$. Empirical results show that our algorithm works well.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

counterfactualMulti-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-agent Reinforcement Learning Accelerated MCMC on Multiscale Inversion Problem

2020-11-17 · Eric Chung, Yalchin Efendiev, Wing Tat Leung, Sai-Mang Pun 외

In this work, we propose a multi-agent actor-critic reinforcement learning (RL) algorithm to accelerate the multi-level Monte Carlo Markov Chain (MCMC) sampling algorithms. The policies (actors) of the agents are used to…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Coordinated Exploration in Concurrent Reinforcement Learning

2018-02-05 · ICML 2018 7 · Maria Dimakopoulou, Benjamin Van Roy

We consider a team of reinforcement learning agents that concurrently learn to operate in a common environment. We identify three properties - adaptivity, commitment, and diversity - which are necessary for efficient coo…

Diversity reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Self-Play Posterior Sampling Algorithm for Zero-Sum Markov Games

2022-10-04 · Wei Xiong, Han Zhong, Chengshuai Shi, Cong Shen 외

Existing studies on provably efficient algorithms for Markov games (MGs) almost exclusively build on the "optimism in the face of uncertainty" (OFU) principle. This work focuses on a different approach of posterior sampl…

Optimistic Posterior Sampling for Reinforcement Learning with Few Samples and Tight Guarantees

2022-09-28 · Daniil Tiapkin, Denis Belomestny, Daniele Calandriello, Eric Moulines 외

We consider reinforcement learning in an environment modeled by an episodic, finite, stage-dependent Markov decision process of horizon $H$ with $S$ states, and $A$ actions. The performance of an agent is measured by the…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Triadic Dynamics Aware Diffusion Posterior Sampling for Inverse Problems: Optimizing Guidance and Stochasticity Schedules

2026-05-26 · Junseo Bang, Dong Ju Mun, Hoigi Seo, Seongmin Hong 외 arxiv

Generative posterior sampling using diffusion models has emerged as a dominant paradigm for solving inverse problems in imaging, which usually consists of three main components: data consistency (DC) guidance, classifier…

Reinforcement Learning