Hierarchy Through Composition with Multitask LMDPs

2017-08-01 · ICML 2017 8 · Andrew M. Saxe, Adam C. Earle, Benjamin Rosman

Hierarchical architectures are critical to the scalability of reinforcement learning methods. Most current hierarchical frameworks execute actions serially, with macro-actions comprising sequences of primitive actions. We propose a novel alternative to these control hierarchies based on concurrent execution of many actions in parallel. Our scheme exploits the guaranteed concurrent compositionality provided by the linearly solvable Markov decision process (LMDP) framework, which naturally enables a learning agent to draw on several macro-actions simultaneously to solve new tasks. We introduce the Multitask LMDP module, which maintains a parallel distributed representation of tasks and may be stacked to form deep hierarchies abstracted in space and time.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical Linearly-Solvable Markov Decision Problems

2016-03-10 · Anders Jonsson, Vicenç Gómez

We present a hierarchical reinforcement learning framework that formulates each task in the hierarchy as a special type of Markov decision process for which the Bellman equation is linear and has analytical solution. Pro…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

RL in Latent MDPs is Tractable: Online Guarantees via Off-Policy Evaluation

2024-06-03 · Jeongyeol Kwon, Shie Mannor, Constantine Caramanis, Yonathan Efroni

In many real-world decision problems there is partially observed, hidden or latent information that remains fixed throughout an interaction. Such decision problems can be modeled as Latent Markov Decision Processes (LMDP…

LEMMA Off-policy evaluation

Incremental Hierarchical Reinforcement Learning with Multitask LMDPs

2018-09-27 · Adam C Earle, Andrew M Saxe, Benjamin Rosman

Exploration is a well known challenge in Reinforcement Learning. One principled way of overcoming this challenge is to find a hierarchical abstraction of the base problem and explore at these higher levels, rather than i…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchy through Composition with Linearly Solvable Markov Decision Processes

2016-12-08 · Andrew M. Saxe, Adam Earle, Benjamin Rosman

Hierarchical architectures are critical to the scalability of reinforcement learning methods. Current hierarchical frameworks execute actions serially, with macro-actions comprising sequences of primitive actions. We pro…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Compositional Transfer in Hierarchical Reinforcement Learning

2019-06-26 · Markus Wulfmeier, Abbas Abdolmaleki, Roland Hafner, Jost Tobias Springenberg 외

The successful application of general reinforcement learning algorithms to real-world robotics applications is often limited by their high data requirements. We introduce Regularized Hierarchical Policy Optimization (RHP…

General Reinforcement Learning Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+3