Full Gradient Deep Reinforcement Learning for Average-Reward Criterion

2023-04-07 · Tejas Pagare, Vivek Borkar, Konstantin Avrachenkov

We extend the provably convergent Full Gradient DQN algorithm for discounted reward Markov decision processes from Avrachenkov et al. (2021) to average reward problems. We experimentally compare widely used RVI Q-Learning with recently proposed Differential Q-Learning in the neural function approximation setting with Full Gradient DQN and DQN. We also extend this to learn Whittle indices for Markovian restless multi-armed bandits. We observe a better convergence rate of the proposed Full Gradient variant across different tasks.

📄 PDF Abstract BibTeX arXiv:2304.03729

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningMulti-Armed BanditsQ-Learningreinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Off-Policy Average Reward Actor-Critic with Deterministic Policy Search

2023-05-20 · Naman Saxena, Subhojyoti Khastigir, Shishir Kolathaya, Shalabh Bhatnagar

The average reward criterion is relatively less studied as most existing works in the Reinforcement Learning literature consider the discounted reward criterion. There are few recent works that present on-policy average …

MuJoCo

On-Policy Deep Reinforcement Learning for the Average-Reward Criterion

2021-06-14 · Yiming Zhang, Keith W. Ross

We develop theory and algorithms for average-reward on-policy Reinforcement Learning (RL). We first consider bounding the difference of the long-term average reward for two policies. We show that previous work based on t…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learning

2024-08-04

In this paper, we propose an off-policy deep reinforcement learning (DRL) method utilizing the average reward criterion. While most existing DRL methods employ the discounted reward criterion, this can potentially lead t…

Inverse Reinforcement Learning with the Average Reward Criterion

2023-05-24 · NeurIPS 2023 11

We study the problem of Inverse Reinforcement Learning (IRL) with an average-reward criterion. The goal is to recover an unknown policy and a reward function when the agent only has samples of states and actions from an …

MuJoCo reinforcement-learning Reinforcement Learning

Hierarchical Average Reward Policy Gradient Algorithms

2019-11-20 · Akshay Dharmavaram, Matthew Riemer, Shalabh Bhatnagar

Option-critic learning is a general-purpose reinforcement learning (RL) framework that aims to address the issue of long term credit assignment by leveraging temporal abstractions. However, when dealing with extended tim…

Reinforcement Learning Reinforcement Learning (RL)