The Eigenoption-Critic Framework

2017-12-11 · Miao Liu, Marlos C. Machado, Gerald Tesauro, Murray Campbell

Eigenoptions (EOs) have been recently introduced as a promising idea for generating a diverse set of options through the graph Laplacian, having been shown to allow efficient exploration. Despite its initial promising results, a couple of issues in current algorithms limit its application, namely: (1) EO methods require two separate steps (eigenoption discovery and reward maximization) to learn a control policy, which can incur a significant amount of storage and computation; (2) EOs are only defined for problems with discrete state-spaces and; (3) it is not easy to take the environment's reward function into consideration when discovering EOs. To addresses these issues, we introduce an algorithm termed eigenoption-critic (EOC) based on the Option-critic (OC) framework [Bacon17], a general hierarchical reinforcement learning (RL) algorithm that allows learning the intra-option policies simultaneously with the policy over options. We also propose a generalization of EOC to problems with continuous state-spaces through the Nystr\"om approximation. EOC can also be seen as extending OC to nonstationary settings, where the discovered options are not tailored for a single task.

📄 PDF Abstract BibTeX arXiv:1712.04065

Code (0)

등록된 구현이 없습니다.

Tasks

Efficient ExplorationHierarchical Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

A Study of Value-Aware Eigenoptions

2025-07-12 · Harshil Kotamreddy, Marlos C. Machado arxiv

Options, which impose an inductive bias toward temporal and hierarchical structure, offer a powerful framework for reinforcement learning (RL). While effective in sequential decision-making, they are often handcrafted ra…

Reinforcement Learning

Eigenoption Discovery through the Deep Successor Representation

2017-10-30 · ICLR 2018 1 · Marlos C. Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu 외

Options in reinforcement learning allow agents to hierarchically decompose a task into subtasks, having the potential to speed up learning and planning. However, autonomously learning effective sets of options is still a…

Atari Games Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+2

Reinforcement Learning with Options and State Representation

2024-03-16 · Ayoub Ghriss, Masashi Sugiyama, Alessandro Lazaric

The current thesis aims to explore the reinforcement learning field and build on existing methods to produce improved ones to tackle the problem of learning in high-dimensional and complex environments. It addresses such…

Decision Making Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Reward-Respecting Subtasks for Model-Based Reinforcement Learning

2022-02-07 · Richard S. Sutton, Marlos C. Machado, G. Zacharias Holland, David Szepesvari 외

To achieve the ambitious goals of artificial intelligence, reinforcement learning must include planning with a model of the world that is abstract in state and time. Deep learning has made progress with state abstraction…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Temporal Abstraction in Reinforcement Learning with the Successor Representation

2021-10-12 · Marlos C. Machado, Andre Barreto, Doina Precup, Michael Bowling

Reasoning at multiple levels of temporal abstraction is one of the key attributes of intelligence. In reinforcement learning, this is often modeled through temporally extended courses of actions called options. Options a…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)