Joint Self-supervised Depth and Optical Flow Estimation towards Dynamic Objects
Significant attention has been attracted to deep learning-based depth estimates. Dynamic objects become the most hard problems in inter-frame-supervised depth estimates due to the uncertainty in adjacent frames. Thus, integrating optical flow information with depth estimation is a feasible solution, as the optical flow is an essential motion representation. In this work, we construct a joint inter-frame-supervised depth and optical flow estimation framework, which predicts depths in various motions by minimizing pixel wrap errors in bilateral photometric re-projections and optical vectors. For motion segmentation, we adaptively segment the preliminary estimated optical flow map with large areas of connectivity. In self-supervised depth estimation, different motion regions are predicted independently and then composite into a complete depth. Further, the pose and depth estimations re-synthesize the optical flow maps, serving to compute reconstruction errors with the preliminary predictions. Our proposed joint depth and optical flow estimation outperforms existing depth estimators on the KITTI Depth dataset, both with and without Cityscapes pretraining. Additionally, our optical flow results demonstrate competitive performance on the KITTI Flow 2015 dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth EstimationMotion SegmentationOptical Flow EstimationSimilar Papers 제목 키워드 기반
Unsupervised Joint Learning of Depth, Optical Flow, Ego-motion from Video
Estimating geometric elements such as depth, camera motion, and optical flow from images is an important part of the robot's visual perception. We use a joint self-supervised method to estimate the three geometric elemen…
Depth EstimationOptical Flow EstimationSemantic SegmentationLearning Optical Flow, Depth, and Scene Flow without Real-World Labels
Self-supervised monocular depth estimation enables robots to learn 3D perception from raw video streams. This scalable approach leverages projective geometry and ego-motion to learn via view synthesis, assuming the world…
Autonomous DrivingDepth EstimationMonocular Depth EstimationMulti-Task Learning+2$\mathrm{F^2Depth}$: Self-supervised Indoor Monocular Depth Estimation via Optical Flow Consistency and Feature Map Synthesis
Self-supervised monocular depth estimation methods have been increasingly given much attention due to the benefit of not requiring large, labelled datasets. Such self-supervised methods require high-quality salient featu…
Depth EstimationIndoor Monocular Depth EstimationMonocular Depth EstimationOptical Flow Estimation+1DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
We present an unsupervised learning framework for simultaneously training single-view depth prediction and optical flow estimation models using unlabeled video sequences. Existing unsupervised methods often exploit brigh…
Depth And Camera MotionDepth EstimationDepth PredictionOptical Flow EstimationUFD-PRiME: Unsupervised Joint Learning of Optical Flow and Stereo Depth through Pixel-Level Rigid Motion Estimation
Both optical flow and stereo disparities are image matches and can therefore benefit from joint training. Depth and 3D motion provide geometric rather than photometric information and can further improve optical flow. Ac…
Motion EstimationOptical Flow Estimation