Benchmarking Batch Deep Reinforcement Learning Algorithms

2019-10-03 · Scott Fujimoto, Edoardo Conti, Mohammad Ghavamzadeh, Joelle Pineau

Widely-used deep reinforcement learning algorithms have been shown to fail in the batch setting--learning from a fixed data set without interaction with the environment. Following this result, there have been several papers showing reasonable performances under a variety of environments and batch settings. In this paper, we benchmark the performance of recent off-policy and batch reinforcement learning algorithms under unified settings on the Atari domain, with data generated by a single partially-trained behavioral policy. We find that under these conditions, many of these algorithms underperform DQN trained online with the same amount of data, as well as the partially-trained behavioral policy. To introduce a strong baseline, we adapt the Batch-Constrained Q-learning algorithm to a discrete-action setting, and show it outperforms all existing algorithms at this task.

📄 PDF Abstract BibTeX arXiv:1910.01708

Code (5)

MLforHealth/rl_representations pytorch

SwarajPawar/Discrete-BCQ pytorch

maziarg/PrivAttack-BCQ pytorch

sfujim/BCQ pytorch

thxsxth/POMDP_RLSepsis pytorch

Tasks

BenchmarkingDeep Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Benchmarking Sample Selection Strategies for Batch Reinforcement Learning

2021-09-29 · Yuwei Fu, Di wu, Benoit Boulet

Training sample section techniques, such as prioritized experience replay (PER), have been recognized as of significant importance for online reinforcement learning algorithms. Efficient sample selection can help further…

Benchmarking Imitation Learning reinforcement-learning Reinforcement Learning+1

Multi-batch Reinforcement Learning via Sample Transfer and Imitation Learning

2021-09-29 · Di wu, Tianyu Li, David Meger, Michael Jenkin 외

Reinforcement learning (RL), especially deep reinforcement learning, has achieved impressive performance on different control tasks. Unfortunately, most online reinforcement learning algorithms require a large number of …

continuous-control Continuous Control Deep Reinforcement Learning Imitation Learning+4

Simulation-Based Benchmarking of Reinforcement Learning Agents for Personalized Retail Promotions

2024-05-16 · Yu Xia, Sriram Narayanamoorthy, Zhengyuan Zhou, Joshua Mabry

The development of open benchmarking platforms could greatly accelerate the adoption of AI agents in retail. This paper presents comprehensive simulations of customer shopping behaviors for the purpose of benchmarking re…

Benchmarking Reinforcement Learning (RL)

Off-Policy Deep Reinforcement Learning without Exploration

2018-12-07 · Scott Fujimoto, David Meger, Doina Precup

Many practical applications of reinforcement learning constrain agents to learn from a fixed batch of data which has already been gathered, without offering further possibility for data collection. In this paper, we demo…

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+2

Where Off-Policy Deep Reinforcement Learning Fails

2018-09-27 · Scott Fujimoto, David Meger, Doina Precup

This work examines batch reinforcement learning--the task of maximally exploiting a given batch of off-policy data, without further data collection. We demonstrate that due to errors introduced by extrapolation, standard…