G2-MonoDepth: A General Framework of Generalized Depth Inference from Monocular RGB+X Data
Monocular depth inference is a fundamental problem for scene perception of robots. Specific robots may be equipped with a camera plus an optional depth sensor of any type and located in various scenes of different scales, whereas recent advances derived multiple individual sub-tasks. It leads to additional burdens to fine-tune models for specific robots and thereby high-cost customization in large-scale industrialization. This paper investigates a unified task of monocular depth inference, which infers high-quality depth maps from all kinds of input raw data from various robots in unseen scenes. A basic benchmark G2-MonoDepth is developed for this task, which comprises four components: (a) a unified data representation RGB+X to accommodate RGB plus raw depth with diverse scene scale/semantics, depth sparsity ([0%, 100%]) and errors (holes/noises/blurs), (b) a novel unified loss to adapt to diverse depth sparsity/errors of input raw data and diverse scales of output scenes, (c) an improved network to well propagate diverse scene scales from input to output, and (d) a data augmentation pipeline to simulate all types of real artifacts in raw depth maps for training. G2-MonoDepth is applied in three sub-tasks including depth estimation, depth completion with different sparsity, and depth enhancement in unseen scenes, and it always outperforms SOTA baselines on both real-world data and synthetic data.
Code (1)
Tasks
Data AugmentationDepth CompletionDepth EstimationSimilar Papers 제목 키워드 기반
Real-time Monocular Depth Estimation on Embedded Systems
Depth sensing is of paramount importance for unmanned aerial and autonomous vehicles. Nonetheless, contemporary monocular depth estimation methods employing complex deep neural networks within Convolutional Neural Networ…
Autonomous VehiclesDecoderDepth EstimationMonocular Depth EstimationRealMonoDepth: Self-Supervised Monocular Depth Estimation for General Scenes
We present a generalised self-supervised learning approach for monocular estimation of the real depth across scenes with diverse depth ranges from 1--100s of meters. Existing supervised methods for monocular depth estima…
Depth EstimationMonocular Depth EstimationSelf-Supervised LearningMonSter: Marry Monodepth to Stereo Unleashes Power
Stereo matching recovers depth from image correspondences. Existing methods struggle to handle ill-posed regions with limited matching cues, such as occlusions and textureless areas. To address this, we propose MonSter, …
Depth EstimationMonocular Depth EstimationStereo MatchingZero-shot GeneralizationSparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion
Estimating scene geometry from data obtained with cost-effective sensors is key for robots and self-driving cars. In this paper, we study the problem of predicting dense depth from a single RGB image (monodepth) with opt…
Depth EstimationDepth PredictionPredictionSelf-Driving Cars+1Full Surround Monodepth from Multiple Cameras
Self-supervised monocular depth and ego-motion estimation is a promising approach to replace or supplement expensive depth sensors such as LiDAR for robotics applications like autonomous driving. However, most research i…
Autonomous DrivingMotion Estimation