D^2USt3R: Enhancing 3D Reconstruction with 4D Pointmaps for Dynamic Scenes
We address the task of 3D reconstruction in dynamic scenes, where object motions degrade the quality of previous 3D pointmap regression methods, such as DUSt3R, originally designed for static 3D scene reconstruction. Although these methods provide an elegant and powerful solution in static settings, they struggle in the presence of dynamic motions that disrupt alignment based solely on camera poses. To overcome this, we propose D^2USt3R that regresses 4D pointmaps that simultaneiously capture both static and dynamic 3D scene geometry in a feed-forward manner. By explicitly incorporating both spatial and temporal aspects, our approach successfully encapsulates spatio-temporal dense correspondence to the proposed 4D pointmaps, enhancing downstream tasks. Extensive experimental evaluations demonstrate that our proposed approach consistently achieves superior reconstruction performance across various datasets featuring complex motions.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Reconstruction3D Scene ReconstructionSimilar Papers 제목 키워드 기반
G3T Up! Gravity Aligned Coordinate Frames Simplify Pointmap Processing
Modern feed-forward 3D reconstruction methods like VGGT predict pixel-aligned pointmaps in camera-centric coordinate frames. However, this choice of coordinate frame is not always optimal. We propose instead to predict p…
3D ReconstructionContinuous 3D Perception Model with Persistent State
We present a unified framework capable of solving a broad range of 3D tasks. Our approach features a stateful recurrent model that continuously updates its state representation with each new observation. Given a stream o…
3D ReconstructionmodelDriv3R: Learning Dense 4D Reconstruction for Autonomous Driving
Realtime 4D reconstruction for dynamic scenes remains a crucial challenge for autonomous driving perception. Most existing methods rely on depth estimation through self-supervision or multi-modality sensor fusion. In thi…
4D reconstructionAutonomous DrivingDepth EstimationSensor FusionCan Video Diffusion Model Reconstruct 4D Geometry?
Reconstructing dynamic 3D scenes (i.e., 4D geometry) from monocular video is an important yet challenging problem. Conventional multiview geometry-based approaches often struggle with dynamic motion, whereas recent learn…
4D reconstructionmodelOptical Flow EstimationC4D: 4D Made from 3D through Dual Correspondences
Recovering 4D from monocular video, which jointly estimates dynamic geometry and camera poses, is an inevitably challenging problem. While recent pointmap-based 3D reconstruction methods (e.g., DUSt3R) have made great pr…
Camera Pose Estimation3D ReconstructionDepth EstimationPoint Tracking