Zero-Sum Positional Differential Games as a Framework for Robust Reinforcement Learning: Deep Q-Learning Approach

2024-05-03 · Anton Plaksin, Vitaly Kalev

Robust Reinforcement Learning (RRL) is a promising Reinforcement Learning (RL) paradigm aimed at training robust to uncertainty or disturbances models, making them more efficient for real-world applications. Following this paradigm, uncertainty or disturbances are interpreted as actions of a second adversarial agent, and thus, the problem is reduced to seeking the agents' policies robust to any opponent's actions. This paper is the first to propose considering the RRL problems within the positional differential game theory, which helps us to obtain theoretically justified intuition to develop a centralized Q-learning approach. Namely, we prove that under Isaacs's condition (sufficiently general for real-world dynamical systems), the same Q-function can be utilized as an approximate solution of both minimax and maximin Bellman equations. Based on these results, we present the Isaacs Deep Q-Network algorithms and demonstrate their superiority compared to other baseline RRL and Multi-Agent RL algorithms in various environments.

📄 PDF Abstract BibTeX arXiv:2405.02044

Code (0)

등록된 구현이 없습니다.

Tasks

Q-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process

2026-06-27 · Congde Hu, Zhuo Jin, Danping Li, Lin Xu arxiv

To address parameter misspecification and sudden structural environmental changes in conventional stochastic differential game (SDG) frameworks, this paper introduces a distributional control approach that characterizes …

Reinforcement Learning

Covariance steering in zero-sum linear-quadratic two-player differential games

2019-09-12

We formulate a new class of two-person zero-sum differential games, in a stochastic setting, where a specification on a target terminal state distribution is imposed on the players. We address such added specification by…

Vocal Bursts Valence Prediction

Combining Deep Reinforcement Learning and Search for Imperfect-Information Games

2020-07-27 · NeurIPS 2020 12 · Noam Brown, Anton Bakhtin, Adam Lerer, Qucheng Gong

The combination of deep reinforcement learning and search at both training and test time is a powerful paradigm that has led to a number of successes in single-agent settings and perfect-information games, best exemplifi…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Minimax Strikes Back

2020-12-19 · Quentin Cohen-Solal, Tristan Cazenave

Deep Reinforcement Learning reaches a superhuman level of play in many complete information games. The state of the art algorithm for learning with zero knowledge is AlphaZero. We take another approach, Ath\'enan, which …

Deep Reinforcement Learning GPU reinforcement-learning Reinforcement Learning (RL)

Understanding Dynamics of Adam in Zero-Sum Games: An ODE Approach

2026-05-19 · Yi Feng, Weiming Ou, Xiao Wang arxiv

The remarkable success of the Adam in training neural networks has naturally led to the widespread use of its descent-ascent counterpart, Adam-DA, for solving zero-sum games. Despite its popularity in practice, a rigorou…