Web Stereo Video Supervision for Depth Prediction from Dynamic Scenes
We present a fully data-driven method to compute depth from diverse monocular video sequences that contain large amounts of non-rigid objects, e.g., people. In order to learn reconstruction cues for non-rigid scenes, we introduce a new dataset consisting of stereo videos scraped in-the-wild. This dataset has a wide variety of scene types, and features large amounts of nonrigid objects, especially people. From this, we compute disparity maps to be used as supervision to train our approach. We propose a loss function that allows us to generate a depth prediction even with unknown camera intrinsics and stereo baselines in the dataset. We validate the use of large amounts of Internet video by evaluating our method on existing video datasets with depth supervision, including SINTEL, and KITTI, and show that our approach generalizes better to natural scenes.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth EstimationDepth PredictionSimilar Papers 제목 키워드 기반
DynamicStereo: Consistent Dynamic Depth from Stereo Videos
We consider the problem of reconstructing a dynamic scene observed from a stereo camera. Most existing methods for depth from stereo treat different stereo frames independently, leading to temporally inconsistent depth p…
StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation
Recent video depth estimation methods achieve great performance by following the paradigm of image depth estimation, i.e., typically fine-tuning pre-trained video diffusion models with massive data. However, we argue tha…
Depth EstimationStereo MatchingGated Stereo: Joint Depth Estimation from Gated and Wide-Baseline Active Stereo Cues
We propose Gated Stereo, a high-resolution and long-range depth estimation technique that operates on active gated stereo images. Using active and high dynamic range passive captures, Gated Stereo exploits multi-view cue…
Depth EstimationDepth PredictionStructure-Guided Ranking Loss for Single Image Depth Prediction
Single image depth prediction is a challenging task due to its ill-posed nature and challenges with capturing ground truth for supervision. Large-scale disparity data generated from stereo photos and 3D videos is a promi…
Depth EstimationDepth PredictionMonocular Depth EstimationPredictionEmpowering Dynamic Urban Navigation with Stereo and Mid-Level Vision
The success of foundation models in language and vision motivated research in fully end-to-end robot navigation foundation models (NFMs). NFMs directly map monocular visual input to control actions and ignore mid-level v…
Spatial ReasoningDepth EstimationRobot Navigation