Value-guided action planning with JEPA world models
Building deep learning models that can reason about their environment requires capturing its underlying dynamics. Joint-Embedded Predictive Architectures (JEPA) provide a promising framework to model such dynamics by learning representations and predictors through a self-supervised prediction objective. However, their ability to support effective action planning remains limited. We propose an approach to enhance planning with JEPA world models by shaping their representation space so that the negative goal-conditioned value function for a reaching cost in a given environment is approximated by a distance (or quasi-distance) between state embeddings. We introduce a practical method to enforce this constraint during training and show that it leads to significantly improved planning performance compared to standard JEPA models on simple control tasks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Policy-Guided World Model Planning for Language-Conditioned Visual Navigation
Navigating to a visually specified goal given natural language instructions remains a fundamental challenge in embodied AI. Existing approaches either rely on reactive policies that struggle with long-horizon planning, o…
Visual NavigationWA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving
Video Joint Embedding Predictive Architecture (V-JEPA) learns powerful spatiotemporal representations from video through self-supervised latent feature prediction. However, V-JEPA is built around random-mask completion a…
Autonomous DrivingFF-JEPA: Long-Horizon Planning in World Models with Latent Planners
Joint Embedding Predictive Architectures (JEPAs) have shown promising world modeling capabilities, enabling planning in latent space by optimizing action trajectories using methods like the Cross-Entropy Method (CEM). Th…
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
A major challenge for modern AI is to learn to understand the world and learn to act largely by observation. This paper explores a self-supervised approach that combines internet-scale video data with a small amount of i…
Action AnticipationLarge Language ModelQuestion AnsweringSelf-Supervised Learning+1AdaJEPA: An Adaptive Latent World Model
Latent world models enable planning from high-dimensional observations by predicting future states in a compact latent space. However, these models are typically kept frozen at test time: when their predictions become in…
Test-time Adaptation