Causal-JEPA: Learning World Models through Object-Level Latent Masking

2026-02-11 · Heejeong Nam, Quentin Le Lidec, Lucas Maes, Yann LeCun, Randall Balestriero arxiv

World models require robust relational understanding to support prediction, reasoning, and control. While object-centric representations provide a useful abstraction, they are not sufficient to capture interaction-dependent dynamics. We therefore propose C-JEPA, a simple and flexible object-centric world model that extends masked joint embedding prediction from image patches to object-centric representations. By masking object-level latents and requiring each masked object state to be inferred from the surrounding context, C-JEPA imposes structured partial observability during training, creating counterfactual-like prediction queries that discourage shortcut solutions and make interaction-dependent prediction necessary under the learning objective. Empirically, C-JEPA leads to consistent gains in visual question answering, with an absolute improvement of about 20% in counterfactual reasoning over the same architecture without object-level masking. On agent control tasks, C-JEPA enables substantially more efficient planning by using only 1% of the total latent input features required by patch-based world models, while achieving comparable performance. Finally, we provide a formal analysis demonstrating that object-level masking induces useful inductive bias by controlling observability. Our code is available at https://github.com/galilai-group/cjepa.

📄 PDF Abstract BibTeX arXiv:2602.11389

Code (0)

등록된 구현이 없습니다.

Tasks

Visual Question Answering

Entity-Centric World Models: Interaction-Aware Masking for Causal Video Prediction

2026-05-14 · Santosh Kumar Paidi arxiv

Learning predictive world models from unlabelled video is a foundational challenge in artificial intelligence. While Joint Embedding Predictive Architectures (JEPA) have set new benchmarks in semantic classification, the…

Video Prediction

A Generalization Theory for JEPA-Based World Models

2026-06-25 · Jingyi Cui, Qi Zhang, Hongwei Wen, Yisen Wang arxiv

Joint Embedding Predictive Architectures (JEPAs) have recently emerged as a promising paradigm for world modeling by learning predictive dynamics in a latent space rather than generating future observations at the input …

Graph Learning

FactorJEPA: Factorizing Monolithic Futures into Layout-Agent-Interaction Channels for Crowded and Chaotic Global South Urban Worlds

2026-08-02 · Kapil Wanaskar, Gaytri Jena, Aman Chadha, Vinija Jain 외 arxiv

World models have attracted significant attention for their ability to capture and predict the structure and dynamics of the physical world. In this emerging landscape, Joint Embedding Predictive Architectures (JEPA) off…

LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics

2026-08-27 · Lukas Kuhn, Lucas Maes, Giuseppe Serra, Quentin Le Lidec 외 arxiv

Video carries the temporal structure of the physical world, yet learning representations from it has remained computationally expensive: prevailing self-supervised methods either prevent representation collapse through a…

JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures

2026-02-19 · Ariel Larey, Elay Dahan, Amit Bleiweiss, Raizy Kellerman 외 arxiv

Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local syntax, these generative paradigms priori…