Curiosity Driven Exploration of Learned Disentangled Goal Spaces

2018-07-04 · Adrien Laversanne-Finot, Alexandre Péré, Pierre-Yves Oudeyer

Intrinsically motivated goal exploration processes enable agents to autonomously sample goals to explore efficiently complex environments with high-dimensional continuous actions. They have been applied successfully to real world robots to discover repertoires of policies producing a wide diversity of effects. Often these algorithms relied on engineered goal spaces but it was recently shown that one can use deep representation learning algorithms to learn an adequate goal space in simple environments. However, in the case of more complex environments containing multiple objects or distractors, an efficient exploration requires that the structure of the goal space reflects the one of the environment. In this paper we show that using a disentangled goal space leads to better exploration performances than an entangled goal space. We further show that when the representation is disentangled, one can leverage it by sampling goals that maximize learning progress in a modular manner. Finally, we show that the measure of learning progress, used to drive curiosity-driven exploration, can be used simultaneously to discover abstract independently controllable features of the environment.

📄 PDF Abstract BibTeX arXiv:1807.01521

Code (1)

flowersteam/Curiosity_Driven_Goal_Exploration 공식 구현 pytorch

Tasks

DiversityEfficient ExplorationRepresentation Learning

GeoExplorer: Active Geo-localization with Curiosity-Driven Exploration

2025-07-31 · Li Mi, Manon Bechaz, Zeming Chen, Antoine Bosselut 외 arxiv

Active Geo-localization (AGL) is the task of localizing a goal, represented in various modalities (e.g., aerial images, ground-level images, or text), within a predefined search area. Current methods approach AGL as a go…

Reinforcement Learning

Curiosity-driven Exploration by Self-supervised Prediction

2017-05-15 · ICML 2017 8 · Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, Trevor Darrell

In many real-world scenarios, rewards extrinsic to the agent are extremely sparse, or absent altogether. In such cases, curiosity can serve as an intrinsic reward signal to enable the agent to explore its environment and…

Prediction Unsupervised Reinforcement Learning

Curiosity-Driven Multi-Criteria Hindsight Experience Replay

2019-06-09 · John B. Lanier, Stephen Mcaleer, Pierre Baldi

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as sta…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Curriculum-guided Hindsight Experience Replay

2019-12-01 · NeurIPS 2019 12 · Meng Fang, Tianyi Zhou, Yali Du, Lei Han 외

In off-policy deep reinforcement learning, it is usually hard to collect sufficient successful experiences with sparse rewards to learn from. Hindsight experience replay (HER) enables an agent to learn from failures by t…

Deep Reinforcement Learning Diversity Reinforcement Learning

See, Hear, Explore: Curiosity via Audio-Visual Association

2020-07-07 · NeurIPS 2020 12 · Victoria Dean, Shubham Tulsiani, Abhinav Gupta

Exploration is one of the core challenges in reinforcement learning. A common formulation of curiosity-driven exploration uses the difference between the real future and the future predicted by a learned model. However, …

Efficient Exploration