Goal-Conditioned Supervised Learning with Sub-Goal Prediction

2023-05-17 · Tom Jurgenson, Aviv Tamar

Recently, a simple yet effective algorithm -- goal-conditioned supervised-learning (GCSL) -- was proposed to tackle goal-conditioned reinforcement-learning. GCSL is based on the principle of hindsight learning: by observing states visited in previously executed trajectories and treating them as attained goals, GCSL learns the corresponding actions via supervised learning. However, GCSL only learns a goal-conditioned policy, discarding other information in the process. Our insight is that the same hindsight principle can be used to learn to predict goal-conditioned sub-goals from the same trajectory. Based on this idea, we propose Trajectory Iterative Learner (TraIL), an extension of GCSL that further exploits the information in a trajectory, and uses it for learning to predict both actions and sub-goals. We investigate the settings in which TraIL can make better use of the data, and discover that for several popular problem settings, replacing real goals in GCSL with predicted TraIL sub-goals allows the agent to reach a greater set of goal states using the exact same data as GCSL, thereby improving its overall performance.

📄 PDF Abstract BibTeX arXiv:2305.10171

Code (0)

등록된 구현이 없습니다.

Tasks

Prediction

Rethinking Goal-conditioned Supervised Learning and Its Connection to Offline RL

2022-02-09 · ICLR 2022 4 · Rui Yang, Yiming Lu, Wenzhe Li, Hao Sun 외

Solving goal-conditioned tasks with sparse rewards using self-supervised learning is promising because of its simplicity and stability over current reinforcement learning (RL) algorithms. A recent work, called Goal-Condi…

Offline RL Reinforcement Learning (RL)Self-Supervised Learning

TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations

2024-07-11 · Junik Bae, Kwanyoung Park, Youngwoon Lee

Unsupervised goal-conditioned reinforcement learning (GCRL) is a promising paradigm for developing diverse robotic skills without external supervision. However, existing unsupervised GCRL methods often struggle to cover …

Self-supervised Learning of Distance Functions for Goal-Conditioned Reinforcement Learning

2019-07-05 · Srinivas Venkattaramanujam, Eric Crawford, Thang Doan, Doina Precup

Goal-conditioned policies are used in order to break down complex reinforcement learning (RL) problems by using subgoals, which can be defined either in state space or in a latent feature space. This can increase the eff…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Self-Supervised Learning

MGDA: Model-based Goal Data Augmentation for Offline Goal-conditioned Weighted Supervised Learning

2024-12-16 · Xing Lei, Xuetao Zhang, Donglin Wang

Recently, a state-of-the-art family of algorithms, known as Goal-Conditioned Weighted Supervised Learning (GCWSL) methods, has been introduced to tackle challenges in offline goal-conditioned reinforcement learning (RL).…

Data Augmentation Reinforcement Learning (RL)

Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning

2021-04-11 · Jinxin Liu, Donglin Wang, Qiangxing Tian, Zhengyu Chen

It is of significance for an agent to learn a widely applicable and general-purpose policy that can achieve diverse goals including images and text descriptions. Considering such perceptually-specific goals, the frontier…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)