Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration

2026-05-21 · Lily Goli, Justin Kerr, Daniele Reda, Alec Jacobson, Andrea Tagliasacchi, Angjoo Kanazawa arxiv

Exploration is a prerequisite for learning useful behaviors in sparse-reward, long-horizon tasks, particularly within 3D environments. Curiosity-driven reinforcement learning addresses this via intrinsic rewards derived from the mismatch between the agent's predictive model of the world and reality. However, translating this intrinsic motivation to complex, photorealistic environments remains difficult, as agents can become trapped in local loops and receive fresh rewards for revisiting forgotten states. In this work, we demonstrate that this failure stems from a lack of spatial persistence and episodic context. We show that effective curiosity requires a model of the world that is persistent and continuously updated, paired with an agent that maintains an episodic trajectory history to navigate toward novel regions. We achieve this using an online 3D reconstruction as a persistent model of the world, while the agent policy is parameterized as a sequence model over RGB observations to maintain episodic context. This design enables effective exploration during training while allowing the agent to navigate using solely RGB frames at deployment. Trained purely via curiosity on HM3D, our agent outperforms RL-based active mapping baselines and generalizes zero-shot to Gibson and AI-generated worlds. Our end-to-end policy enables efficient adaptation to downstream tasks, such as apple picking and image-goal navigation, outperforming from-scratch baselines. Please see video results at https://recuriosity.github.io/.

📄 PDF Abstract BibTeX arXiv:2605.22814

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning 3D Reconstruction

Now I Remember! Episodic Memory For Reinforcement Learning

2018-01-01 · ICLR 2018 1 · Ricky Loynd, Matthew Hausknecht, Lihong Li, Li Deng

Humans rely on episodic memory constantly, in remembering the name of someone they met 10 minutes ago, the plot of a movie as it unfolds, or where they parked the car. Endowing reinforcement learning agents with episodic…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Web World Models

2025-12-29 · Jichen Feng, Yifan Zhang, Chenggong Zhang, Yifu Lu 외 arxiv

Language agents increasingly require persistent worlds in which they can act, remember, and learn. Existing approaches sit at two extremes: conventional web frameworks provide reliable but fixed contexts backed by databa…

A model of semantic completion in generative episodic memory

2021-11-26 · Zahra Fayyaz, Aya Altamimi, Sen Cheng, Laurenz Wiskott

Many different studies have suggested that episodic memory is a generative process, but most computational models adopt a storage view. In this work, we propose a computational model for generative episodic memory. It is…

Hippocampus

REMem: Reasoning with Episodic Memory in Language Agent

2026-02-13 · Yiheng Shu, Saisri Padmaja Jonnalagedda, Xiang Gao, Bernal Jiménez Gutiérrez 외 arxiv

Humans excel at remembering concrete experiences along spatiotemporal contexts and performing reasoning across those events, i.e., the capacity for episodic memory. In contrast, memory in language agents remains mainly s…

Beyond Episodic Evaluation: Memory Architectural Bottlenecks in Sequential Embodied Question Answering

2026-07-23 · Zikui Cai, Kaushal Janga, Tan Dat Dao, Seungjae Lee 외 arxiv

Embodied question answering (EQA) is traditionally evaluated under an episodic formulation, where agents solve each task independently and reset internal state between episodes. However, real-world robots operate continu…

Question Answering