Algorithms for Batch Hierarchical Reinforcement Learning

2016-03-29 · Tiancheng Zhao, Mohammad Gowayyed

Hierarchical Reinforcement Learning (HRL) exploits temporal abstraction to solve large Markov Decision Processes (MDP) and provide transferable subtask policies. In this paper, we introduce an off-policy HRL algorithm: Hierarchical Q-value Iteration (HQI). We show that it is possible to effectively learn recursive optimal policies for any valid hierarchical decomposition of the original MDP, given a fixed dataset collected from a flat stochastic behavioral policy. We first formally prove the convergence of the algorithm for tabular MDP. Then our experiments on the Taxi domain show that HQI converges faster than a flat Q-value Iteration and enjoys easy state abstraction. Also, we demonstrate that our algorithm is able to learn optimal policies for different hierarchical structures from the same fixed dataset, which enables model comparison without recollecting data.

📄 PDF Abstract BibTeX arXiv:1603.08869

Code (0)

등록된 구현이 없습니다.

Tasks

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)valid

Online Baum-Welch algorithm for Hierarchical Imitation Learning

2021-03-22 · Vittorio Giammarino, Ioannis Ch. Paschalidis

The options framework for hierarchical reinforcement learning has increased its popularity in recent years and has made improvements in tackling the scalability problem in reinforcement learning. Yet, most of these recen…

Hierarchical Reinforcement Learning Imitation Learning reinforcement-learning Reinforcement Learning+1

Benchmarking Batch Deep Reinforcement Learning Algorithms

2019-10-03 · Scott Fujimoto, Edoardo Conti, Mohammad Ghavamzadeh, Joelle Pineau

Widely-used deep reinforcement learning algorithms have been shown to fail in the batch setting--learning from a fixed data set without interaction with the environment. Following this result, there have been several pap…

Benchmarking Deep Reinforcement Learning Q-Learning reinforcement-learning+2

Multi-batch Reinforcement Learning via Sample Transfer and Imitation Learning

2021-09-29 · Di wu, Tianyu Li, David Meger, Michael Jenkin 외

Reinforcement learning (RL), especially deep reinforcement learning, has achieved impressive performance on different control tasks. Unfortunately, most online reinforcement learning algorithms require a large number of …

continuous-control Continuous Control Deep Reinforcement Learning Imitation Learning+4

SaLinA: Sequential Learning of Agents

2021-10-15 · Ludovic Denoyer, Alfredo De la Fuente, Song Duong, Jean-Baptiste Gaya 외

SaLinA is a simple library that makes implementing complex sequential learning models easy, including reinforcement learning algorithms. It is built as an extension of PyTorch: algorithms coded with \SALINA{} can be unde…

reinforcement-learning Reinforcement Learning (RL)

Off-Policy Deep Reinforcement Learning without Exploration

2018-12-07 · Scott Fujimoto, David Meger, Doina Precup

Many practical applications of reinforcement learning constrain agents to learn from a fixed batch of data which has already been gathered, without offering further possibility for data collection. In this paper, we demo…

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+2