Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learning

2023-04-03 · Tongzhou Wang, Antonio Torralba, Phillip Isola, Amy Zhang

In goal-reaching reinforcement learning (RL), the optimal value function has a particular geometry, called quasimetric structure. This paper introduces Quasimetric Reinforcement Learning (QRL), a new RL method that utilizes quasimetric models to learn optimal value functions. Distinct from prior approaches, the QRL objective is specifically designed for quasimetrics, and provides strong theoretical recovery guarantees. Empirically, we conduct thorough analyses on a discretized MountainCar environment, identifying properties of QRL and its advantages over alternatives. On offline and online goal-reaching benchmarks, QRL also demonstrates improved sample efficiency and performance, across both state-based and image-based observations.

📄 PDF Abstract BibTeX arXiv:2304.01203

Code (1)

quasimetric-learning/quasimetric-rl 공식 구현 pytorch

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Offline Goal-conditioned Reinforcement Learning with Quasimetric Representations

2025-09-24 · Vivek Myers, Bill Chunyuan Zheng, Benjamin Eysenbach, Sergey Levine arxiv

Approaches for goal-conditioned reinforcement learning (GCRL) often use learned state representations to extract goal-reaching policies. Two frameworks for representation structure have yielded particularly effective GCR…

Representation Learning Reinforcement Learning Contrastive Learning

Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement Learning

2025-12-12 · Vittorio Giammarino, Ahmed H. Qureshi arxiv

Goal-Conditioned Reinforcement Learning (GCRL) mitigates the difficulty of reward design by framing tasks as goal reaching rather than maximizing hand-crafted reward signals. In this setting, the optimal goal-conditioned…

Reinforcement Learning

Intrinsic-Energy Joint Embedding Predictive Architectures Induce Quasimetric Spaces

2026-02-12 · Anthony Kobanda, Waris Radji arxiv

Joint-Embedding Predictive Architectures (JEPAs) aim to learn representations by predicting target embeddings from context embeddings, inducing a scalar compatibility energy in a latent space. In contrast, Quasimetric Re…

Reinforcement Learning

On the Learning and Learnability of Quasimetrics

2022-06-30 · Tongzhou Wang, Phillip Isola

Our world is full of asymmetries. Gravity and wind can make reaching a place easier than coming back. Social artifacts such as genealogy charts and citation graphs are inherently directed. In reinforcement learning and c…

Q-Learning Reinforcement Learning (RL)

On the Learning of Quasimetrics

2021-09-29 · ICLR 2022 4 · Tongzhou Wang, Phillip Isola