LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels

2026-03-13 · Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, Randall Balestriero arxiv

Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces, yet existing methods remain fragile, relying on complex multi-term losses, exponential moving averages, pre-trained encoders, or auxiliary supervision to avoid representation collapse. In this work, we introduce LeWorldModel (LeWM), the first JEPA that trains stably end-to-end from raw pixels using only two loss terms: a next-embedding prediction loss and a regularizer enforcing Gaussian-distributed latent embeddings. This reduces tunable loss hyperparameters from six to one compared to the only existing end-to-end alternative. With ~15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models while remaining competitive across diverse 2D and 3D control tasks. Beyond control, we show that LeWM's latent space encodes meaningful physical structure through probing of physical quantities. Surprise evaluation confirms that the model reliably detects physically implausible events.

📄 PDF Abstract BibTeX arXiv:2603.19312

Code (0)

등록된 구현이 없습니다.

Fast LeWorldModel

2026-06-24 · Yuntian Gao, Xiangyu Xu arxiv

Joint-Embedding Predictive Architectures (JEPAs), including recent LeWorldModel (LeWM), have become a promising foundation for reconstruction-free visual world models. For visual planning, however, LeWM evaluates candida…

Sub-JEPA: Subspace Gaussian Regularization for Stable End-to-End World Models

2026-05-10 · Kai Zhao, Dongliang Nie, Yuchen Lin, Zhehan Luo 외 arxiv

Joint-Embedding Predictive Architectures (JEPAs) provide a simpleframework for learning world models by predicting future latent representations.However, JEPA training is subject to a bias-variance tradeoff.Without suffi…

BiJEPA: Bi-directional Joint Embedding Predictive Architecture for Symmetric Representation Learning

2026-02-10 · Yongchao Huang arxiv

Self-Supervised Learning (SSL) has shifted from pixel-level reconstruction to latent space prediction, spearheaded by the Joint Embedding Predictive Architecture (JEPA). While effective, standard JEPA models typically re…

Self-Supervised Learning Representation Learning

Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization

2026-09-09 · Andy Zeyi Liu, Haoran Sun, Lucas Baker, Randall Balestriero 외 arxiv

Joint-Embedding Predictive Architecture (JEPA) world models learn a compact latent representation of the world that supports prediction and planning, but their capability to learn physics and generate physically realisti…

SCALE: State-Calibrated Latent Embeddings for JEPA Planning in the Right Geometry

2026-08-17 · Jiaming Hu, Yan Zheng, Tian Wang arxiv

Joint-embedding predictive world models plan by scoring predicted terminal embeddings against a goal embedding using a cost defined on the representation itself. Two prominent strategies for obtaining non-collapsed repre…