Unsupervised Learning of Depth, Camera Pose and Optical Flow from Monocular Video

2022-05-19 · Dipan Mandal, Abhilash Jain

We propose DFPNet -- an unsupervised, joint learning system for monocular Depth, Optical Flow and egomotion (Camera Pose) estimation from monocular image sequences. Due to the nature of 3D scene geometry these three components are coupled. We leverage this fact to jointly train all the three components in an end-to-end manner. A single composite loss function -- which involves image reconstruction-based loss for depth & optical flow, bidirectional consistency checks and smoothness loss components -- is used to train the network. Using hyperparameter tuning, we are able to reduce the model size to less than 5% (8.4M parameters) of state-of-the-art DFP models. Evaluation on KITTI and Cityscapes driving datasets reveals that our model achieves results comparable to state-of-the-art in all of the three tasks, even with the significantly smaller model size.

📄 PDF Abstract BibTeX arXiv:2205.09821

Code (0)

등록된 구현이 없습니다.

Tasks

Camera Pose Estimation Image Reconstruction Optical Flow Estimation Pose Estimation

DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency

2018-09-05 · ECCV 2018 9 · Yuliang Zou, Zelun Luo, Jia-Bin Huang

We present an unsupervised learning framework for simultaneously training single-view depth prediction and optical flow estimation models using unlabeled video sequences. Existing unsupervised methods often exploit brigh…

Depth And Camera Motion Depth Estimation Depth Prediction Optical Flow Estimation

Feature-Level Collaboration: Joint Unsupervised Learning of Optical Flow, Stereo Depth and Camera Motion

2021-06-19 · CVPR 2021 1 · Cheng Chi, Qingjie Wang, Tianyu Hao, Peng Guo 외

Precise estimation of optical flow, stereo depth and camera motion are important for the real-world 3D scene understanding and visual perception. Since the three tasks are tightly coupled with the inherent 3D geometr…

Camera Pose Estimation Decoder Depth And Camera Motion Depth Estimation+5

Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometry

2020-03-02 · arXiv 2020 3 · Guangming Wang, Chi Zhang, Hesheng Wang, Jingchuan Wang 외

In autonomous driving, monocular sequences contain lots of information. Monocular depth estimation, camera ego-motion estimation and optical flow estimation in consecutive frames are high-profile concerns recently. By an…

3D geometry Autonomous Driving Depth And Camera Motion Depth Estimation+3

UnOS: Unified Unsupervised Optical-Flow and Stereo-Depth Estimation by Watching Videos

2019-06-01 · CVPR 2019 6 · Yang Wang, Peng Wang, Zhenheng Yang, Chenxu Luo 외

In this paper, we propose UnOS, an unified system for unsupervised optical flow and stereo depth estimation using convolutional neural network (CNN) by taking advantages of their inherent geometrical consistency based on…

Depth Estimation Motion Segmentation Optical Flow Estimation Stereo Depth Estimation+1

Learning Residual Flow as Dynamic Motion from Stereo Videos

2019-09-16 · Seokju Lee, Sunghoon Im, Stephen Lin, In So Kweon

We present a method for decomposing the 3D scene flow observed from a moving stereo rig into stationary scene elements and dynamic object motion. Our unsupervised learning framework jointly reasons about the camera motio…

Depth And Camera Motion Motion Estimation Optical Flow Estimation Stereo Matching+2