홈 › Papers

Eigenoption Discovery through the Deep Successor Representation

2017-10-30 · ICLR 2018 1 · Marlos C. Machado, Clemens Rosenbaum, Xiaoxiao Guo, Miao Liu, Gerald Tesauro, Murray Campbell

Options in reinforcement learning allow agents to hierarchically decompose a task into subtasks, having the potential to speed up learning and planning. However, autonomously learning effective sets of options is still a major challenge in the field. In this paper we focus on the recently introduced idea of using representation learning methods to guide the option discovery process. Specifically, we look at eigenoptions, options obtained from representations that encode diffusive information flow in the environment. We extend the existing algorithms for eigenoption discovery to settings with stochastic transitions and in which handcrafted features are not available. We propose an algorithm that discovers eigenoptions while learning non-linear state representations from raw pixels. It exploits recent successes in the deep reinforcement learning literature and the equivalence between proto-value functions and the successor representation. We use traditional tabular domains to provide intuition about our approach and Atari 2600 games to demonstrate its potential.

📄 PDF Abstract BibTeX arXiv:1710.11089

Code (1)

mklissa/dceo jax

Tasks

Atari GamesDeep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Representation Learning

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Temporal Abstraction in Reinforcement Learning with the Successor Representation

2021-10-12 · Marlos C. Machado, Andre Barreto, Doina Precup, Michael Bowling

Reasoning at multiple levels of temporal abstraction is one of the key attributes of intelligence. In reinforcement learning, this is often modeled through temporally extended courses of actions called options. Options a…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

The Eigenoption-Critic Framework

2017-12-11 · Miao Liu, Marlos C. Machado, Gerald Tesauro, Murray Campbell

Eigenoptions (EOs) have been recently introduced as a promising idea for generating a diverse set of options through the graph Laplacian, having been shown to allow efficient exploration. Despite its initial promising re…

Efficient Exploration Hierarchical Reinforcement Learning Reinforcement Learning Reinforcement Learning (RL)

A Study of Value-Aware Eigenoptions

2025-07-12 · Harshil Kotamreddy, Marlos C. Machado arxiv

Options, which impose an inductive bias toward temporal and hierarchical structure, offer a powerful framework for reinforcement learning (RL). While effective in sequential decision-making, they are often handcrafted ra…

Reinforcement Learning

Successor Options : An Option Discovery Algorithm for Reinforcement Learning

2018-09-27 · Manan Tomar*, Rahul Ramesh*, Balaraman Ravindran

Hierarchical Reinforcement Learning is a popular method to exploit temporal abstractions in order to tackle the curse of dimensionality. The options framework is one such hierarchical framework that models the notion of …

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Successor Options: An Option Discovery Framework for Reinforcement Learning

2019-05-14 · Rahul Ramesh, Manan Tomar, Balaraman Ravindran

The options framework in reinforcement learning models the notion of a skill or a temporally extended sequence of actions. The discovery of a reusable set of skills has typically entailed building options, that navigate …

Navigate reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)