A Game Theoretic Perspective on Model-Based Reinforcement Learning

2020-01-01 · ICML 2020 1 · Aravind Rajeswaran, Igor Mordatch, Vikash Kumar

We illustrate how game theory is a good framework to understand model-based reinforcement learning (MBRL). We point out that a large class of MBRL algorithms can be viewed as a game between two players: (1) a policy player, which attempts to maximize rewards under the learned model; (2) a model player, which attempts to fit the real-world data collected by the policy player. Their goals need not be aligned, and are often conflicting. We show that stable algorithms for MBRL can be derived by considering a Stackelberg game between the two players. This formulation gives rise to two natural schools of MBRL algorithms based on which player is chosen as the leader in the Stackelberg game, and together encapsulate many existing MBRL algorithms. Through experiments on a suite of continuous control tasks, we validate that algorithms based on our framework lead to stable and sample efficient learning.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous ControlModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games

2026-02-11 · Kazuki Ota, Takayuki Osa, Motoki Omura, Tatsuya Harada arxiv

Two-player games such as board games have long been used as traditional benchmarks for reinforcement learning. This work revisits a policy optimization method with reverse Kullback-Leibler regularization and entropy regu…

Reinforcement Learning

Finite-Time Analysis of Q-Value Iteration for General-Sum Stackelberg Games

2026-04-06 · Narim Jeong, Donghwan Lee arxiv

Reinforcement learning has been successful both empirically and theoretically in single-agent settings, but extending these results to multi-agent reinforcement learning in general-sum Markov games remains challenging. T…

Multi-agent Reinforcement Learning

Multi-Objective Reinforcement Learning with Max-Min Criterion: A Game-Theoretic Approach

2025-10-23 · Woohyeon Byeon, Giseung Park, Jongseong Chae, Amir Leshem 외 arxiv

In this paper, we propose a provably convergent and practical framework for multi-objective reinforcement learning with max-min criterion. From a game-theoretic perspective, we reformulate max-min multi-objective reinfor…

Reinforcement Learning

From Game-theoretic Multi-agent Log Linear Learning to Reinforcement Learning

2018-02-07 · Mohammadhosein Hasanbeig, Lacra Pavel

The main focus of this paper is on enhancement of two types of game-theoretic learning algorithms: log-linear learning and reinforcement learning. The standard analysis of log-linear learning needs a highly structured en…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Game-Theoretical Analysis of Reviewer Rewards in Peer-Review Journal Systems: Analysis and Experimental Evaluation using Deep Reinforcement Learning

2023-05-20 · Minhyeok Lee

In this paper, we navigate the intricate domain of reviewer rewards in open-access academic publishing, leveraging the precision of mathematics and the strategic acumen of game theory. We conceptualize the prevailing vou…

Deep Reinforcement Learning Navigate reinforcement-learning Reinforcement Learning