3D Distillation: Improving Self-Supervised Monocular Depth Estimation on Reflective Surfaces
Self-supervised monocular depth estimation (SSMDE) aims at predicting the dense depth maps of monocular images, by learning to minimize a photometric loss using spatially neighboring image pairs during training. While SSMDE offers a significant scalability advantage over supervised approaches, it performs poorly on reflective surfaces as the photometric constancy assumption of the photometric loss is violated. We note that the appearance of reflective surfaces is view-dependent and often there are views of such surfaces in the training data that are not contaminated by strong specular reflections. Thus, reflective surfaces can be accurately reconstructed by aggregating the predicted depth of these views. Motivated by this observation, we propose 3D distillation: a novel training framework that utilizes the projected depth of reconstructed reflective surfaces to generate reasonably accurate depth pseudo-labels. To identify those surfaces automatically, we employ an uncertainty-guided depth fusion method, combining the smoother and more accurate projected depth on reflective surfaces and the detailed predicted depth elsewhere. In our experiments using the ScanNet and 7-Scenes datasets, we show that 3D distillation not only significantly improves the prediction accuracy, especially on the problematic surfaces, but also that it generalizes well over various underlying network architectures and to new datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth EstimationMonocular Depth EstimationSimilar Papers 제목 키워드 기반
Revealing the Reciprocal Relations Between Self-Supervised Stereo and Monocular Depth Estimation
Current self-supervised depth estimation algorithms mainly focus on either stereo or monocular only, neglecting the reciprocal relations between them. In this paper, we propose a simple yet effective framework to imp…
Depth EstimationMonocular Depth EstimationStereo MatchingFisheyeDistill: Self-Supervised Monocular Depth Estimation with Ordinal Distillation for Fisheye Cameras
In this paper, we deal with the problem of monocular depth estimation for fisheye cameras in a self-supervised manner. A known issue of self-supervised depth estimation is that it suffers in low-light/over-exposure condi…
Depth EstimationMonocular Depth EstimationExcavating the Potential Capacity of Self-Supervised Monocular Depth Estimation
Self-supervised methods play an increasingly important role in monocular depth estimation due to their great potential and low annotation cost. To close the gap with supervised methods, recent works take advantage of ext…
Data AugmentationDepth EstimationMonocular Depth EstimationSemantic SegmentationSelf-Supervised Monocular Depth Estimation with Self-Reference Distillation and Disparity Offset Refinement
Monocular depth estimation plays a fundamental role in computer vision. Due to the costly acquisition of depth ground truth, self-supervised methods that leverage adjacent frames to establish a supervisory signal have em…
Depth EstimationMonocular Depth EstimationSUB-Depth: Self-distillation and Uncertainty Boosting Self-supervised Monocular Depth Estimation
We propose SUB-Depth, a universal multi-task training framework for self-supervised monocular depth estimation (SDE). Depth models trained with SUB-Depth outperform the same models trained in a standard single-task SDE f…
Depth EstimationImage ReconstructionMonocular Depth Estimation