2T-UNET: A Two-Tower UNet with Depth Clues for Robust Stereo Depth Estimation
Stereo correspondence matching is an essential part of the multi-step stereo depth estimation process. This paper revisits the depth estimation problem, avoiding the explicit stereo matching step using a simple two-tower convolutional neural network. The proposed algorithm is entitled as 2T-UNet. The idea behind 2T-UNet is to replace cost volume construction with twin convolution towers. These towers have an allowance for different weights between them. Additionally, the input for twin encoders in 2T-UNet are different compared to the existing stereo methods. Generally, a stereo network takes a right and left image pair as input to determine the scene geometry. However, in the 2T-UNet model, the right stereo image is taken as one input and the left stereo image along with its monocular depth clue information, is taken as the other input. Depth clues provide complementary suggestions that help enhance the quality of predicted scene geometry. The 2T-UNet surpasses state-of-the-art monocular and stereo depth estimation methods on the challenging Scene flow dataset, both quantitatively and qualitatively. The architecture performs incredibly well on complex natural scenes, highlighting its usefulness for various real-time applications. Pretrained weights and code will be made readily available.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth EstimationStereo Depth EstimationStereo MatchingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Novel 3D-UNet Deep Learning Framework Based on High-Dimensional Bilateral Grid for Edge Consistent Single Image Depth Estimation
The task of predicting smooth and edge-consistent depth maps is notoriously difficult for single image depth estimation. This paper proposes a novel Bilateral Grid based 3D convolutional neural network, dubbed as 3DBG-UN…
Depth EstimationDepth PredictionSemantic SegmentationClamNet: Using contrastive learning with variable depth Unets for medical image segmentation
Unets have become the standard method for semantic segmentation of medical images, along with fully convolutional networks (FCN). Unet++ was introduced as a variant of Unet, in order to solve some of the problems facing …
Computed Tomography (CT)Contrastive LearningImage SegmentationMedical Image Segmentation+1UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation
The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations: (1) their optimal depth is apriori unknow…
Computed Tomography (CT)DecoderImage SegmentationInstance Segmentation+3SAR-UNet: Small Attention Residual UNet for Explainable Nowcasting Tasks
The accuracy and explainability of data-driven nowcasting models are of great importance in many socio-economic sectors reliant on weather-dependent decision making. This paper proposes a novel architecture called Small …
Decision MakingDecoderEnd-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving
Inspired by the UNet architecture of semantic image segmentation, we propose a lightweight UNet using depthwise separable convolutions (DSUNet) for end-to-end learning of lane detection and path prediction (PP) in autono…
Autonomous DrivingImage SegmentationLane DetectionSemantic Segmentation