Reinforcement Learning with Options and State Representation

2024-03-16 · Ayoub Ghriss, Masashi Sugiyama, Alessandro Lazaric

The current thesis aims to explore the reinforcement learning field and build on existing methods to produce improved ones to tackle the problem of learning in high-dimensional and complex environments. It addresses such goals by decomposing learning tasks in a hierarchical fashion known as Hierarchical Reinforcement Learning. We start in the first chapter by getting familiar with the Markov Decision Process framework and presenting some of its recent techniques that the following chapters use. We then proceed to build our Hierarchical Policy learning as an answer to the limitations of a single primitive policy. The hierarchy is composed of a manager agent at the top and employee agents at the lower level. In the last chapter, which is the core of this thesis, we attempt to learn lower-level elements of the hierarchy independently of the manager level in what is known as the "Eigenoption". Based on the graph structure of the environment, Eigenoptions allow us to build agents that are aware of the geometric and dynamic properties of the environment. Their decision-making has a special property: it is invariant to symmetric transformations of the environment, allowing as a consequence to greatly reduce the complexity of the learning task.

📄 PDF Abstract BibTeX arXiv:2403.10855

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

AWARE We propose to theoretically and empirically examine the effect of incorporating weighting schemes into walk-aggregating GNNs. To this end, we propose a simple, interpretable, and…

Successor Options: An Option Discovery Framework for Reinforcement Learning

2019-05-14 · Rahul Ramesh, Manan Tomar, Balaraman Ravindran

The options framework in reinforcement learning models the notion of a skill or a temporally extended sequence of actions. The discovery of a reusable set of skills has typically entailed building options, that navigate …

Navigate reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Eigenoption Discovery through the Deep Successor Representation

2017-10-30 · ICLR 2018 1 · Marlos C. Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu 외

Options in reinforcement learning allow agents to hierarchically decompose a task into subtasks, having the potential to speed up learning and planning. However, autonomously learning effective sets of options is still a…

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+2

Temporal Abstraction in Reinforcement Learning with the Successor Representation

2021-10-12 · Marlos C. Machado, Andre Barreto, Doina Precup, Michael Bowling

Reasoning at multiple levels of temporal abstraction is one of the key attributes of intelligence. In reinforcement learning, this is often modeled through temporally extended courses of actions called options. Options a…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Autonomous Option Invention for Continual Hierarchical Reinforcement Learning and Planning

2024-12-20 · Rashmeet Kaur Nayyar, Siddharth Srivastava

Abstraction is key to scaling up reinforcement learning (RL). However, autonomously learning abstract state and action representations to enable transfer and generalization remains a challenging open problem. This paper …

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Successor Options : An Option Discovery Algorithm for Reinforcement Learning

2018-09-27 · Manan Tomar*, Rahul Ramesh*, Balaraman Ravindran

Hierarchical Reinforcement Learning is a popular method to exploit temporal abstractions in order to tackle the curse of dimensionality. The options framework is one such hierarchical framework that models the notion of …