Multi-scale Predictive Representations for Goal-conditioned Reinforcement Learning

2026-05-10 · Valliappan Chidambaram Adaikkappan, David Meger, Sai Rajeswar, Pietro Mazzaglia arxiv

This paper investigates robust representation learning in offline goal-conditioned reinforcement learning (GCRL). Particularly in sparse reward scenarios, learning representations that align state and goal latents is a challenge that frequently culminates in representation divergence where the encoder drifts toward a low-dimensional, goal-agnostic subspace that destabilizes policy learning. We address this issue by showing that an agent must acquire a fundamental understanding of its environment across multiple scales, from local physical dynamics to long-horizon goal-directed structure. Building on this insight, we propose Ms.PR, a framework that leverages multi-scale predictive supervision to enforce goal-directed alignment within the latent space. We demonstrate that Ms.PR leads to improved representation quality and strong performance on both vision and state-based tasks. Furthermore, we show that our approach is exceptionally resilient under realistic, challenging data regimes, maintaining state-of-the-art performance across a wide variety of tasks, trajectory stitching scenarios, and extreme noise conditions.

📄 PDF Abstract BibTeX arXiv:2605.09364

Code (0)

등록된 구현이 없습니다.

Tasks

Representation Learning Reinforcement Learning

Goal-Conditioned Predictive Coding for Offline Reinforcement Learning

2023-07-07 · NeurIPS 2023 11

Recent work has demonstrated the effectiveness of formulating decision making as supervised learning on offline-collected trajectories. Powerful sequence models, such as GPT or BERT, are often employed to encode the traj…

Decision Making Offline RL reinforcement-learning Reinforcement Learning

Contrastive Difference Predictive Coding

2023-10-31 · Chongyi Zheng, Ruslan Salakhutdinov, Benjamin Eysenbach

Predicting and reasoning about the future lie at the heart of many time-series questions. For example, goal-conditioned reinforcement learning can be viewed as learning representations to predict which states are likely …

Representation Learning Time Series

Act2Goal: From World Model To General Goal-conditioned Policy

2025-12-29 · Pengfei Zhou, Liliang Chen, Shengcong Chen, Di Chen 외 arxiv

Specifying robotic manipulation tasks in a manner that is both expressive and precise remains a central challenge. While visual goals provide a compact and unambiguous task specification, existing goal-conditioned polici…

Zero-shot Generalization

Why Goal-Conditioned Reinforcement Learning Works: Relation to Dual Control

2025-12-06 · Nathan P. Lawrence, Ali Mesbah arxiv

Goal-conditioned reinforcement learning (RL) concerns the problem of training an agent to maximize the probability of reaching target goal states. This paper presents an analysis of the goal-conditioned setting based on …

Reinforcement Learning

Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learning

2022-11-01 · Riashat Islam, Hongyu Zang, Anirudh Goyal, Alex Lamb 외

Goal-conditioned reinforcement learning (RL) is a promising direction for training agents that are capable of solving multiple tasks and reach a diverse set of objectives. How to \textit{specify} and \textit{ground} thes…

reinforcement-learning Reinforcement Learning (RL)