Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance
Self-supervised monocular depth estimation presents a powerful method to obtain 3D scene information from single camera images, which is trainable on arbitrary image sequences without requiring depth labels, e.g., from a LiDAR sensor. In this work we present a new self-supervised semantically-guided depth estimation (SGDepth) method to deal with moving dynamic-class (DC) objects, such as moving cars and pedestrians, which violate the static-world assumptions typically made during training of such models. Specifically, we propose (i) mutually beneficial cross-domain training of (supervised) semantic segmentation and self-supervised depth estimation with task-specific network heads, (ii) a semantic masking scheme providing guidance to prevent moving DC objects from contaminating the photometric loss, and (iii) a detection method for frames with non-moving DC objects, from which the depth of DC objects can be learned. We demonstrate the performance of our method on several benchmarks, in particular on the Eigen split, where we exceed all baselines without test-time refinement.
Code (1)
Tasks
Depth EstimationMonocular Depth EstimationSemantic SegmentationSimilar Papers 제목 키워드 기반
$S^3$Net: Semantic-Aware Self-supervised Depth Estimation with Monocular Videos and Synthetic Data
Solving depth estimation with monocular cameras enables the possibility of widespread use of cameras as low-cost depth estimation sensors in applications such as autonomous driving and robotics. However, learning such a …
Autonomous DrivingDepth EstimationDomain AdaptationPanoptic SegmentationS³Net: Semantic-Aware Self-supervised Depth Estimation with Monocular Videos and Synthetic Data
Solving depth estimation with monocular cameras enables the possibility of widespread use of cameras as low-cost depth estimation sensors in applications such as autonomous driving and robotics. In order to learn such a …
Autonomous DrivingDepth EstimationDomain AdaptationSelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning
Monocular depth estimation in the wild inherently predicts depth up to an unknown scale. To resolve scale ambiguity issue, we present a learning algorithm that leverages monocular simultaneous localization and mapping (S…
Depth EstimationMonocular Depth EstimationRobot NavigationSelf-Supervised Learning+1RealMonoDepth: Self-Supervised Monocular Depth Estimation for General Scenes
We present a generalised self-supervised learning approach for monocular estimation of the real depth across scenes with diverse depth ranges from 1--100s of meters. Existing supervised methods for monocular depth estima…
Depth EstimationMonocular Depth EstimationSelf-Supervised LearningFusionDepth: Complement Self-Supervised Monocular Depth Estimation with Cost Volume
Multi-view stereo depth estimation based on cost volume usually works better than self-supervised monocular depth estimation except for moving objects and low-textured surfaces. So in this paper, we propose a multi-frame…
Depth EstimationMonocular Depth EstimationStereo Depth Estimation