Temporally Consistent Depth Prediction with Flow-Guided Memory Units

2019-09-16 · Chanho Eom, Hyunjong Park, Bumsub Ham

Predicting depth from a monocular video sequence is an important task for autonomous driving. Although it has advanced considerably in the past few years, recent methods based on convolutional neural networks (CNNs) discard temporal coherence in the video sequence and estimate depth independently for each frame, which often leads to undesired inconsistent results over time. To address this problem, we propose to memorize temporal consistency in the video sequence, and leverage it for the task of depth prediction. To this end, we introduce a two-stream CNN with a flow-guided memory module, where each stream encodes visual and temporal features, respectively. The memory module, implemented using convolutional gated recurrent units (ConvGRUs), inputs visual and temporal features sequentially together with optical flow tailored to our task. It memorizes trajectories of individual features selectively and propagates spatial information over time, enforcing a long-term temporal consistency to prediction results. We evaluate our method on the KITTI benchmark dataset in terms of depth prediction accuracy, temporal consistency and runtime, and achieve a new state of the art. We also provide an extensive experimental analysis, clearly demonstrating the effectiveness of our approach to memorizing temporal consistency for depth prediction.

📄 PDF Abstract BibTeX arXiv:1909.07074

Code (0)

등록된 구현이 없습니다.

Tasks

Autonomous Driving Depth EstimationDepth PredictionOptical Flow Estimation Prediction

Consistent Depth of Moving Objects in Video

2021-08-02 · Zhoutong Zhang, Forrester Cole, Richard Tucker, William T. Freeman 외

We present a method to estimate depth of a dynamic scene, containing arbitrary moving objects, from an ordinary video captured with a moving camera. We seek a geometrically and temporally consistent solution to this unde…

Depth Estimation Depth Prediction Prediction Rolling Shutter Correction+1

Learning Depth from Monocular Videos Using Synthetic Data: A Temporally-Consistent Domain Adaptation Approach

2019-07-16 · Yipeng Mou, Mingming Gong, Huan Fu, Kayhan Batmanghelich 외

Majority of state-of-the-art monocular depth estimation methods are supervised learning approaches. The success of such approaches heavily depends on the high-quality depth labels which are expensive to obtain. Some rece…

Depth Estimation Depth Prediction Domain Adaptation Monocular Depth Estimation+3

Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation

2021-10-15 · Patrick Ruhkamp, Daoyi Gao, Hanzhi Chen, Nassir Navab 외

Inferring geometrically consistent dense 3D scenes across a tuple of temporally consecutive images remains challenging for self-supervised monocular depth prediction pipelines. This paper explores how the increasingly po…

Depth Estimation Depth Prediction Monocular Depth Estimation

Veritatem Dies Aperit- Temporally Consistent Depth Prediction Enabled by a Multi-Task Geometric and Semantic Scene Understanding Approach

2019-03-26 · Amir Atapour-Abarghouei, Toby P. Breckon

Robust geometric and semantic scene understanding is ever more important in many real-world applications such as autonomous driving and robotic navigation. In this paper, we propose a multi-task learning-based approach c…

Autonomous Driving Depth Completion Depth Estimation Depth Prediction+4

Veritatem Dies Aperit - Temporally Consistent Depth Prediction Enabled by a Multi-Task Geometric and Semantic Scene Understanding Approach

2019-06-01 · CVPR 2019 6 · Amir Atapour-Abarghouei, Toby P. Breckon