Exploration in Reinforcement Learning with Deep Covering Options

2020-05-01 · ICLR 2020 1 · Yuu Jinnai, Jee Won Park, Marlos C. Machado, George Konidaris

While many option discovery methods have been proposed to accelerate exploration in reinforcement learning, they are often heuristic. Recently, covering options was proposed to discover a set of options that provably reduce the upper bound of the environment's cover time, a measure of the difficulty of exploration. Covering options are computed using the eigenvectors of the graph Laplacian, but they are constrained to tabular tasks and are not applicable to tasks with large or continuous state-spaces. We introduce deep covering options, an online method that extends covering options to large state spaces, automatically discovering task-agnostic options that encourage exploration. We evaluate our method in several challenging sparse-reward domains and we show that our approach identifies less explored regions of the state-space and successfully generates options to visit these regions, substantially improving both the exploration and the total accumulated reward.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A Study of Value-Aware Eigenoptions

2025-07-12 · Harshil Kotamreddy, Marlos C. Machado arxiv

Options, which impose an inductive bias toward temporal and hierarchical structure, offer a powerful framework for reinforcement learning (RL). While effective in sequential decision-making, they are often handcrafted ra…

Reinforcement Learning

Discovering Options for Exploration by Minimizing Cover Time

2019-03-02 · Yuu Jinnai, Jee Won Park, David Abel, George Konidaris

One of the main challenges in reinforcement learning is solving tasks with sparse reward. We show that the difficulty of discovering a distant rewarding state in an MDP is bounded by the expected cover time of a random w…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-agent Deep Covering Skill Discovery

2022-10-07 · Jiayu Chen, Marina Haliem, Tian Lan, Vaneet Aggarwal

The use of skills (a.k.a., options) can greatly accelerate exploration in reinforcement learning, especially when only sparse reward signals are available. While option discovery methods have been proposed for individual…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Successor Options : An Option Discovery Algorithm for Reinforcement Learning

2018-09-27 · Manan Tomar*, Rahul Ramesh*, Balaraman Ravindran

Hierarchical Reinforcement Learning is a popular method to exploit temporal abstractions in order to tackle the curse of dimensionality. The options framework is one such hierarchical framework that models the notion of …

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep Laplacian-based Options for Temporally-Extended Exploration

2023-01-26 · Martin Klissarov, Marlos C. Machado

Selecting exploratory actions that generate a rich stream of experience for better learning is a fundamental challenge in reinforcement learning (RL). An approach to tackle this problem consists in selecting actions acco…

Reinforcement Learning (RL)