Policy Iteration for Pareto-Optimal Policies in Stochastic Stackelberg Games

2024-05-07 · Mikoto Kudo, Yohei Akimoto

In general-sum stochastic games, a stationary Stackelberg equilibrium (SSE) does not always exist, in which the leader maximizes leader's return for all the initial states when the follower takes the best response against the leader's policy. Existing methods of determining the SSEs require strong assumptions to guarantee the convergence and the coincidence of the limit with the SSE. Moreover, our analysis suggests that the performance at the fixed points of these methods is not reasonable when they are not SSEs. Herein, we introduced the concept of Pareto-optimality as a reasonable alternative to SSEs. We derive the policy improvement theorem for stochastic games with the best-response follower and propose an iterative algorithm to determine the Pareto-optimal policies based on it. Monotone improvement and convergence of the proposed approach are proved, and its convergence to SSEs is proved in a special case.

📄 PDF Abstract BibTeX arXiv:2405.06689

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SSE 설명 없음

Learning Fair Pareto-Optimal Policies in Multi-Objective Reinforcement Learning

2026-06-16 · Umer Siddique, Peilang Li, Yongcan Cao arxiv

Fairness is an important aspect of decision-making in multi-objective reinforcement learning (MORL), where policies must ensure both optimality and equity across multiple, potentially conflicting objectives. While single…

Reinforcement Learning

Self-Play PSRO: Toward Optimal Populations in Two-Player Zero-Sum Games

2022-07-13 · Stephen Mcaleer, JB Lanier, Kevin Wang, Pierre Baldi 외

In competitive two-agent environments, deep reinforcement learning (RL) methods based on the \emph{Double Oracle (DO)} algorithm, such as \emph{Policy Space Response Oracles (PSRO)} and \emph{Anytime PSRO (APSRO)}, itera…

Deep Reinforcement Learning Reinforcement Learning (RL)

Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

2021-07-16 · Bo Pang, Zhong-Ping Jiang

This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Traversing Pareto Optimal Policies: Provably Efficient Multi-Objective Reinforcement Learning

2024-07-24 · Shuang Qiu, Dake Zhang, Rui Yang, Boxiang Lyu 외

This paper investigates multi-objective reinforcement learning (MORL), which focuses on learning Pareto optimal policies in the presence of multiple reward functions. Despite MORL's significant empirical success, there i…

Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning

Deterministic Pareto-Optimal Policy Synthesis for Multi-Objective Reinforcement Learning

2026-06-24 · Aniruddha Joshi, Niklas Lauffer, Sanjit Seshia arxiv

Real-world decision-making often requires balancing multiple conflicting objectives, a challenge that standard Reinforcement Learning (RL) frequently addresses by aggregating rewards into a single scalar signal. While ef…