Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme

2021-03-10 · K. E. Avrachenkov, V. S. Borkar, H. P. Dolhare, K. Patil

We analyze the DQN reinforcement learning algorithm as a stochastic approximation scheme using the o.d.e. (for 'ordinary differential equation') approach and point out certain theoretical issues. We then propose a modified scheme called Full Gradient DQN (FG-DQN, for short) that has a sound theoretical basis and compare it with the original scheme on sample problems. We observe a better performance for FG-DQN.

📄 PDF Abstract BibTeX arXiv:2103.05981

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Full Gradient Deep Reinforcement Learning for Average-Reward Criterion

2023-04-07 · Tejas Pagare, Vivek Borkar, Konstantin Avrachenkov

We extend the provably convergent Full Gradient DQN algorithm for discounted reward Markov decision processes from Avrachenkov et al. (2021) to average reward problems. We experimentally compare widely used RVI Q-Learnin…

Deep Reinforcement Learning Multi-Armed Bandits Q-Learning reinforcement-learning+1

Provably Convergent Two-Timescale Off-Policy Actor-Critic with Function Approximation

2019-11-11 · ICML 2020 1 · Shangtong Zhang, Bo Liu, Hengshuai Yao, Shimon Whiteson

We present the first provably convergent two-timescale off-policy actor-critic algorithm (COF-PAC) with function approximation. Key to COF-PAC is the introduction of a new critic, the emphasis critic, which is trained vi…

Vocal Bursts Valence Prediction

Smoothed Q-learning

2023-03-15 · David Barber

In Reinforcement Learning the Q-learning algorithm provably converges to the optimal solution. However, as others have demonstrated, Q-learning can also overestimate the values and thereby spend too long exploring unhelp…

Q-Learning reinforcement-learning Reinforcement Learning (RL)

Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imaging

2023-11-15 · Marcello Carioni, Subhadip Mukherjee, Hong Ye Tan, Junqi Tang

Unsupervised deep learning approaches have recently become one of the crucial research areas in imaging owing to their ability to learn expressive and powerful reconstruction operators even when paired high-quality train…

Survey

Mirror and Preconditioned Gradient Descent in Wasserstein Space

2024-06-13 · Clément Bonet, Théo Uscidda, Adam David, Pierre-Cyril Aubin-Frankowski 외

As the problem of minimizing functionals on the Wasserstein space encompasses many applications in machine learning, different optimization algorithms on $\mathbb{R}^d$ have received their counterpart analog on the Wasse…