Hierarchical Representation Learning for Markov Decision Processes

2021-06-03 · Lorenzo Steccanella, Simone Totaro, Anders Jonsson

In this paper we present a novel method for learning hierarchical representations of Markov decision processes. Our method works by partitioning the state space into subsets, and defines subtasks for performing transitions between the partitions. We formulate the problem of partitioning the state space as an optimization problem that can be solved using gradient descent given a set of sampled trajectories, making our method suitable for high-dimensional problems with large state spaces. We empirically validate the method, by showing that it can successfully learn a useful hierarchical representation in a navigation domain. Once learned, the hierarchical representation can be used to solve different tasks in the given domain, thus generalizing knowledge across tasks.

📄 PDF Abstract BibTeX arXiv:2106.01655

Code (0)

등록된 구현이 없습니다.

Tasks

Representation Learning

Hierarchy through Composition with Linearly Solvable Markov Decision Processes

2016-12-08 · Andrew M. Saxe, Adam Earle, Benjamin Rosman

Hierarchical architectures are critical to the scalability of reinforcement learning methods. Current hierarchical frameworks execute actions serially, with macro-actions comprising sequences of primitive actions. We pro…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hierarchical Average-Reward Linearly-solvable Markov Decision Processes

2024-07-09 · Guillermo Infante, Anders Jonsson, Vicenç Gómez

We introduce a novel approach to hierarchical reinforcement learning for Linearly-solvable Markov Decision Processes (LMDPs) in the infinite-horizon average-reward setting. Unlike previous work, our approach allows learn…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Incremental Hierarchical Reinforcement Learning with Multitask LMDPs

2018-09-27 · Adam C Earle, Andrew M Saxe, Benjamin Rosman

Exploration is a well known challenge in Reinforcement Learning. One principled way of overcoming this challenge is to find a hierarchical abstraction of the base problem and explore at these higher levels, rather than i…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Omega-Regular Decision Processes

2023-12-14 · Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi 외

Regular decision processes (RDPs) are a subclass of non-Markovian decision processes where the transition and reward functions are guarded by some regular property of the past (a lookback). While RDPs enable intuitive an…

Globally Optimal Hierarchical Reinforcement Learning for Linearly-Solvable Markov Decision Processes

2021-06-29 · Guillermo Infante, Anders Jonsson, Vicenç Gómez

In this work we present a novel approach to hierarchical reinforcement learning for linearly-solvable Markov decision processes. Our approach assumes that the state space is partitioned, and the subtasks consist in movin…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)