Continuous 3D Perception Model with Persistent State

2025-01-21 · CVPR 2025 1 · Qianqian Wang, Yifei Zhang, Aleksander Holynski, Alexei A. Efros, Angjoo Kanazawa

We present a unified framework capable of solving a broad range of 3D tasks. Our approach features a stateful recurrent model that continuously updates its state representation with each new observation. Given a stream of images, this evolving state can be used to generate metric-scale pointmaps (per-pixel 3D points) for each new input in an online fashion. These pointmaps reside within a common coordinate system, and can be accumulated into a coherent, dense scene reconstruction that updates as new images arrive. Our model, called CUT3R (Continuous Updating Transformer for 3D Reconstruction), captures rich priors of real-world scenes: not only can it predict accurate pointmaps from image observations, but it can also infer unseen regions of the scene by probing at virtual, unobserved views. Our method is simple yet highly flexible, naturally accepting varying lengths of images that may be either video streams or unordered photo collections, containing both static and dynamic content. We evaluate our method on various 3D/4D tasks and demonstrate competitive or state-of-the-art performance in each. Project Page: https://cut3r.github.io/

📄 PDF Abstract BibTeX arXiv:2501.12387

Code (0)

등록된 구현이 없습니다.

Tasks

3D Reconstructionmodel

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Adam 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Residual Connection 설명 없음

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

H-RINS: Hierarchical Tightly-coupled Radar-Inertial State Estimation via Smoothing and Mapping

2026-03-14 · Ali Alridha Abdulkarim, Mikhail Litvinov, Dzmitry Tsetserukou arxiv

Millimeter-wave radar enables robust perception in visually degraded environments, yet radar-inertial estimation remains prone to drift: sparse body-frame velocity measurements weakly constrain absolute orientation, leav…

StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding

2026-08-06 · Xichen Zhang, Guankai Li, Yinghao Zhu, Shijian Wang 외 arxiv

Deploying autonomous multimodal agents in continuous, real-world environments requires them to ingest unbounded audio-visual streams and maintain hour-scale memory. However, current evaluations predominantly rely on brie…

Springdrift: An Auditable Persistent Runtime for LLM Agents with Case-Based Memory, Normative Safety, and Ambient Self-Perception

2026-04-06 · Seamus Brady arxiv

We present Springdrift, a persistent runtime for long-lived LLM agents. The system integrates an auditable execution substrate (append-only memory, supervised processes, git-backed recovery), a case-based reasoning memor…

Agents in the Large: Perception-Centered Architecture for Persistent Agents

2026-08-31 · Shihan Dou, Haoxiang Jia, Shichun Liu, Feng Chen 외 arxiv

Cognitive language agents have achieved substantial progress by equipping language models with memory, tools, and decision-making procedures, enabling agents to reason and act in interactive environments. Existing framew…

OpenSPM: An Environment-Transferable Robotic Key Spatial Pose Memory and Closed-Loop High-Frequency Flow-Matching Action Generation Model

2026-06-29 · Iok Tong Lei, Qingchen Xie, Yifan Wang, Yap Ying Jie 외 arxiv

Open-environment tabletop robotic manipulation requires systems to possess semantic understanding, precise geometric pose estimation, and high-frequency action generation. While end-to-end vision-language-action (VLA) mo…

Pose Estimation