Self-supervised Object Motion and Depth Estimation from Video
We present a self-supervised learning framework to estimate the individual object motion and monocular depth from video. We model the object motion as a 6 degree-of-freedom rigid-body transformation. The instance segmentation mask is leveraged to introduce the information of object. Compared with methods which predict dense optical flow map to model the motion, our approach significantly reduces the number of values to be estimated. Our system eliminates the scale ambiguity of motion prediction through imposing a novel geometric constraint loss term. Experiments on KITTI driving dataset demonstrate our system is capable to capture the object motion without external annotation. Our system outperforms previous self-supervised approaches in terms of 3D scene flow prediction, and contribute to the disparity prediction in dynamic area.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth EstimationInstance Segmentationmotion predictionObjectOptical Flow EstimationPredictionSelf-Supervised LearningSemantic SegmentationSimilar Papers 제목 키워드 기반
Mining Supervision for Dynamic Regions in Self-Supervised Monocular Depth Estimation
This paper focuses on self-supervised monocular depth estimation in dynamic scenes trained on monocular videos. Existing methods jointly estimate pixel-wise depth and motion, relying mainly on an image reconstruction los…
Depth EstimationImage ReconstructionMonocular Depth EstimationMotion EstimationDO3D: Self-supervised Learning of Decomposed Object-aware 3D Motion and Depth from Monocular Videos
Although considerable advancements have been attained in self-supervised depth estimation from monocular videos, most existing methods often treat all objects in a video as static entities, which however violates the dyn…
Depth EstimationDisentanglementMotion DisentanglementMotion Estimation+3MAL: Motion-Aware Loss with Temporal and Distillation Hints for Self-Supervised Depth Estimation
Depth perception is crucial for a wide range of robotic applications. Multi-frame self-supervised depth estimation methods have gained research interest due to their ability to leverage large-scale, unlabeled real-world …
Depth EstimationMonocular Depth EstimationJoint Self-supervised Depth and Optical Flow Estimation towards Dynamic Objects
Significant attention has been attracted to deep learning-based depth estimates. Dynamic objects become the most hard problems in inter-frame-supervised depth estimates due to the uncertainty in adjacent frames. Thus, in…
Depth EstimationMotion SegmentationOptical Flow EstimationSelf-Supervised Joint Learning Framework of Depth Estimation via Implicit Cues
In self-supervised monocular depth estimation, the depth discontinuity and motion objects' artifacts are still challenging problems. Existing self-supervised methods usually utilize a single view to train the depth estim…
Depth EstimationMonocular Depth Estimation