Self-supervised Learning of Distance Functions for Goal-Conditioned Reinforcement Learning

2019-07-05 · Srinivas Venkattaramanujam, Eric Crawford, Thang Doan, Doina Precup

Goal-conditioned policies are used in order to break down complex reinforcement learning (RL) problems by using subgoals, which can be defined either in state space or in a latent feature space. This can increase the efficiency of learning by using a curriculum, and also enables simultaneous learning and generalization across goals. A crucial requirement of goal-conditioned policies is to be able to determine whether the goal has been achieved. Having a notion of distance to a goal is thus a crucial component of this approach. However, it is not straightforward to come up with an appropriate distance, and in some tasks, the goal space may not even be known a priori. In this work we learn a distance-to-goal estimate which is computed in terms of the number of actions that would need to be carried out in a self-supervised approach. Our method solves complex tasks without prior domain knowledge in the online setting in three different scenarios in the context of goal-conditioned policies a) the goal space is the same as the state space b) the goal space is given but an appropriate distance is unknown and c) the state space is accessible, but only a subset of the state space represents desired goals, and this subset is known a priori. We also propose a goal-generation mechanism as a secondary contribution.

📄 PDF Abstract BibTeX arXiv:1907.02998

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Self-Supervised Learning

TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations

2024-07-11 · Junik Bae, Kwanyoung Park, Youngwoon Lee

Unsupervised goal-conditioned reinforcement learning (GCRL) is a promising paradigm for developing diverse robotic skills without external supervision. However, existing unsupervised GCRL methods often struggle to cover …

Safe Multi-Agent Navigation guided by Goal-Conditioned Safe Reinforcement Learning

2025-02-25 · Meng Feng, Viraj Parimi, Brian Williams

Safe navigation is essential for autonomous systems operating in hazardous environments. Traditional planning methods excel at long-horizon tasks but rely on a predefined graph with fixed distance metrics. In contrast, s…

Benchmarking Reinforcement Learning (RL)Safe Reinforcement Learning

VLD: Visual Language Goal Distance for Reinforcement Learning Navigation

2025-12-08 · Lazar Milikic, Manthan Patel, Jonas Frey arxiv

Training end-to-end policies from image data to directly predict navigation actions for robotic systems has proven inherently difficult. Existing approaches often suffer from either the sim-to-real gap during policy tran…

Reinforcement Learning

State Representation Learning for Goal-Conditioned Reinforcement Learning

2022-05-04 · Lorenzo Steccanella, Anders Jonsson

This paper presents a novel state representation for reward-free Markov decision processes. The idea is to learn, in a self-supervised manner, an embedding space where distances between pairs of embedded states correspon…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Representation Learning

LEAF: Latent Exploration Along the Frontier

2020-05-21 · Homanga Bharadhwaj, Animesh Garg, Florian Shkurti

Self-supervised goal proposal and reaching is a key component for exploration and efficient policy learning algorithms. Such a self-supervised approach without access to any oracle goal sampling distribution requires dee…