Self-Supervised Surround-View Depth Estimation with Volumetric Feature Fusion
We present a self-supervised depth estimation approach using a unified volumetric feature fusion for surround-view images. Given a set of surround-view images, our method constructs a volumetric feature map by extracting image feature maps from surround-view images and fuse the feature maps into a shared, unified 3D voxel space. The volumetric feature map then can be used for estimating a depth map at each surround view by projecting it into an image coordinate. A volumetric feature contains 3D information at its local voxel coordinate; thus our method can also synthesize a depth map at arbitrary rotated viewpoints by projecting the volumetric feature map into the target viewpoints. Furthermore, assuming static camera extrinsics in the multi-camera system, we propose to estimate a canonical camera motion from the volumetric feature map. Our method leverages 3D spatiotemporal context to learn metric-scale depth and the canonical camera motion in a self-supervised manner. Our method outperforms the prior arts on DDAD and nuScenes datasets, especially estimating more accurate metric-scale depth and consistent depth between neighboring views.
Code (1)
Tasks
Autonomous DrivingDepth EstimationMonocular Depth EstimationSimilar Papers 제목 키워드 기반
SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation
Depth estimation from images serves as the fundamental step of 3D perception for autonomous driving and is an economical alternative to expensive depth sensors like LiDAR. The temporal photometric constraints enables sel…
Autonomous DrivingDepth EstimationMonocular Depth EstimationCross-Vehicle 3D Geometric Consistency for Self-Supervised Surround Depth Estimation on Articulated Vehicles
Surround depth estimation provides a cost-effective alternative to LiDAR for 3D perception in autonomous driving. While recent self-supervised methods explore multi-camera settings to improve scale awareness and scene co…
Autonomous DrivingDepth EstimationA Simple Baseline for Supervised Surround-view Depth Estimation
Depth estimation has been widely studied and serves as the fundamental step of 3D perception for intelligent vehicles. Though significant progress has been made in monocular depth estimation in the past decades, these at…
Autonomous DrivingDepth EstimationMonocular Depth EstimationGeoSurDepth: Harnessing Foundation Model for Spatial Geometry Consistency-Oriented Self-Supervised Surround-View Depth Estimation
Accurate surround-view depth estimation provides a competitive alternative to laser-based sensors and is essential for 3D scene understanding in autonomous driving. While empirical studies have proposed various approache…
Novel View SynthesisImage ReconstructionScene UnderstandingAutonomous DrivingTowards Cross-View-Consistent Self-Supervised Surround Depth Estimation
Depth estimation is a cornerstone for autonomous driving, yet acquiring per-pixel depth ground truth for supervised learning is challenging. Self-Supervised Surround Depth Estimation (SSSDE) from consecutive images offer…
Autonomous DrivingDepth EstimationPose Estimation