Exploration by Learning Diverse Skills through Successor State Measures

2024-06-14 · Paul-Antoine Le Tolguenec, Yann Besse, Florent Teichteil-Konigsbuch, Dennis G. Wilson, Emmanuel Rachelson

The ability to perform different skills can encourage agents to explore. In this work, we aim to construct a set of diverse skills which uniformly cover the state space. We propose a formalization of this search for diverse skills, building on a previous definition based on the mutual information between states and skills. We consider the distribution of states reached by a policy conditioned on each skill and leverage the successor state measure to maximize the difference between these skill distributions. We call this approach LEADS: Learning Diverse Skills through Successor States. We demonstrate our approach on a set of maze navigation and robotic control tasks which show that our method is capable of constructing a diverse set of skills which exhaustively cover the state space without relying on reward or exploration bonuses. Our findings demonstrate that this new formalization promotes more robust and efficient exploration by combining mutual information maximization and exploration bonuses.

📄 PDF Abstract BibTeX arXiv:2406.10127

Code (0)

등록된 구현이 없습니다.

Tasks

Efficient Exploration

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Successor Options : An Option Discovery Algorithm for Reinforcement Learning

2018-09-27 · Manan Tomar*, Rahul Ramesh*, Balaraman Ravindran

Hierarchical Reinforcement Learning is a popular method to exploit temporal abstractions in order to tackle the curse of dimensionality. The options framework is one such hierarchical framework that models the notion of …

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Maximum State Entropy Exploration using Predecessor and Successor Representations

2023-06-26 · NeurIPS 2023 11

Animals have a developed ability to explore that aids them in important tasks such as locating food, exploring for shelter, and finding misplaced items. These exploration skills necessarily track where they have been so …

Efficient Exploration

Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memory

2026-08-17 · Bingxin Xu, Yuzhang Shang, Emilio Ferrara arxiv

Long-horizon robot manipulation chains many contact-rich skills into one multi-stage task. Vision-language-action (VLA) models increasingly master the individual skills, yet the chain still fails: errors compound beyond …

Robot Manipulation

Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Critics

2024-03-15 · Luca Grillotti, Maxence Faldor, Borja G. León, Antoine Cully

A key aspect of intelligence is the ability to demonstrate a broad spectrum of behaviors for adapting to unexpected situations. Over the past decade, advancements in deep reinforcement learning have led to groundbreaking…

continuous-control Continuous Control Deep Reinforcement Learning Diversity+2

Successor Options: An Option Discovery Framework for Reinforcement Learning

2019-05-14 · Rahul Ramesh, Manan Tomar, Balaraman Ravindran

The options framework in reinforcement learning models the notion of a skill or a temporally extended sequence of actions. The discovery of a reusable set of skills has typically entailed building options, that navigate …

Navigate reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)