A Game Theoretic Framework for Model Based Reinforcement Learning

2020-04-16 · Aravind Rajeswaran, Igor Mordatch, Vikash Kumar

Model-based reinforcement learning (MBRL) has recently gained immense interest due to its potential for sample efficiency and ability to incorporate off-policy data. However, designing stable and efficient MBRL algorithms using rich function approximators have remained challenging. To help expose the practical challenges in MBRL and simplify algorithm design from the lens of abstraction, we develop a new framework that casts MBRL as a game between: (1) a policy player, which attempts to maximize rewards under the learned model; (2) a model player, which attempts to fit the real-world data collected by the policy player. For algorithm development, we construct a Stackelberg game between the two players, and show that it can be solved with approximate bi-level optimization. This gives rise to two natural families of algorithms for MBRL based on which player is chosen as the leader in the Stackelberg game. Together, they encapsulate, unify, and generalize many previous MBRL algorithms. Furthermore, our framework is consistent with and provides a clear basis for heuristics known to be important in practice from prior works. Finally, through experiments we validate that our proposed algorithms are highly sample efficient, match the asymptotic performance of model-free policy gradient, and scale gracefully to high-dimensional tasks like dexterous hand manipulation. Additional details and code can be obtained from the project page at https://sites.google.com/view/mbrl-game

📄 PDF Abstract BibTeX arXiv:2004.07804

Code (0)

등록된 구현이 없습니다.

Tasks

Model-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games

2026-02-11 · Kazuki Ota, Takayuki Osa, Motoki Omura, Tatsuya Harada arxiv

Two-player games such as board games have long been used as traditional benchmarks for reinforcement learning. This work revisits a policy optimization method with reverse Kullback-Leibler regularization and entropy regu…

Reinforcement Learning

Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach

2025-10-23 · Woohyeon Byeon, Giseung Park, Jongseong Chae, Amir Leshem 외 arxiv

In this paper, we propose a provably convergent and practical framework for multi-objective reinforcement learning with max-min criterion. From a game-theoretic perspective, we reformulate max-min multi-objective reinfor…

Reinforcement Learning

GARL: Game-Theoretic Reinforcement Learning for Multi-Agent Strategic Prioritisation

2026-06-03 · Yuxiao Ye, Yiwen Zhang, Huiyuan Xie, Yuqin Huang 외 arxiv

LLM-based multi-agent systems are increasingly used for strategic decision-making tasks. In such settings, performance depends not only on individual model capabilities, but also on the policies by which agents interact …

Multi-agent Reinforcement Learning

Multi-agent Inverse Reinforcement Learning for Two-person Zero-sum Games

2014-03-25 · Xiaomin Lin, Peter A. Beling, Randy Cogill

The focus of this paper is a Bayesian framework for solving a class of problems termed multi-agent inverse reinforcement learning (MIRL). Compared to the well-known inverse reinforcement learning (IRL) problem, MIRL is f…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Valence Prediction

PAC Reinforcement Learning Algorithm for General-Sum Markov Games

2020-09-05 · Ashkan Zehfroosh, Herbert G. Tanner

This paper presents a theoretical framework for probably approximately correct (PAC) multi-agent reinforcement learning (MARL) algorithms for Markov games. The paper offers an extension to the well-known Nash Q-learning …

Multi-agent Reinforcement Learning Q-Learning reinforcement-learning Reinforcement Learning+1