Hierarchical Linearly-Solvable Markov Decision Problems

2016-03-10 · Anders Jonsson, Vicenç Gómez

We present a hierarchical reinforcement learning framework that formulates each task in the hierarchy as a special type of Markov decision process for which the Bellman equation is linear and has analytical solution. Problems of this type, called linearly-solvable MDPs (LMDPs) have interesting properties that can be exploited in a hierarchical setting, such as efficient learning of the optimal value function or task compositionality. The proposed hierarchical approach can also be seen as a novel alternative to solving LMDPs with large state spaces. We derive a hierarchical version of the so-called Z-learning algorithm that learns different tasks simultaneously and show empirically that it significantly outperforms the state-of-the-art learning methods in two classical hierarchical reinforcement learning domains: the taxi domain and an autonomous guided vehicle task.

📄 PDF Abstract BibTeX arXiv:1603.03267

Code (0)

등록된 구현이 없습니다.

Tasks

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchy through Composition with Linearly Solvable Markov Decision Processes

2016-12-08 · Andrew M. Saxe, Adam Earle, Benjamin Rosman

Hierarchical architectures are critical to the scalability of reinforcement learning methods. Current hierarchical frameworks execute actions serially, with macro-actions comprising sequences of primitive actions. We pro…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical Average-Reward Linearly-solvable Markov Decision Processes

2024-07-09 · Guillermo Infante, Anders Jonsson, Vicenç Gómez

We introduce a novel approach to hierarchical reinforcement learning for Linearly-solvable Markov Decision Processes (LMDPs) in the infinite-horizon average-reward setting. Unlike previous work, our approach allows learn…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Globally Optimal Hierarchical Reinforcement Learning for Linearly-Solvable Markov Decision Processes

2021-06-29 · Guillermo Infante, Anders Jonsson, Vicenç Gómez

In this work we present a novel approach to hierarchical reinforcement learning for linearly-solvable Markov decision processes. Our approach assumes that the state space is partitioned, and the subtasks consist in movin…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Actor-Critic for Linearly-Solvable Continuous MDP with Partially Known Dynamics

2017-06-04 · Tomoki Nishi, Prashant Doshi, Michael R. James, Danil Prokhorov

In many robotic applications, some aspects of the system dynamics can be modeled accurately while others are difficult to obtain or model. We present a novel reinforcement learning (RL) method for continuous state and ac…

Reinforcement Learning Reinforcement Learning (RL)

Hierarchy Through Composition with Multitask LMDPs

2017-08-01 · ICML 2017 8 · Andrew M. Saxe, Adam C. Earle, Benjamin Rosman

Hierarchical architectures are critical to the scalability of reinforcement learning methods. Most current hierarchical frameworks execute actions serially, with macro-actions comprising sequences of primitive actio…