Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds
Understanding the flow in 3D space of sparsely sampled points between two consecutive time frames is the core stone of modern geometric-driven systems such as VR/AR, Robotics, and Autonomous driving. The lack of real, non-simulated, labeled data for this task emphasizes the importance of self- or un-supervised deep architectures. This work presents a new self-supervised training method and an architecture for the 3D scene flow estimation under occlusions. Here we show that smart multi-layer fusion between flow prediction and occlusion detection outperforms traditional architectures by a large margin for occluded and non-occluded scenarios. We report state-of-the-art results on Flyingthings3D and KITTI datasets for both the supervised and self-supervised training.
Code (1)
Tasks
Autonomous DrivingScene Flow EstimationSelf-supervised Scene Flow EstimationSimilar Papers 제목 키워드 기반
Self-supervised Learning of Occlusion Aware Flow Guided 3D Geometry Perception with Adaptive Cross Weighted Loss from Monocular Videos
Self-supervised deep learning-based 3D scene understanding methods can overcome the difficulty of acquiring the densely labeled ground-truth and have made a lot of advances. However, occlusions and moving objects are sti…
3D geometry3D Geometry PerceptionCamera Pose EstimationOptical Flow Estimation+3RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow
Monocular scene flow estimation aims to recover dense 3D motion from image sequences, yet most existing methods are limited to two-frame inputs, restricting temporal modeling and robustness to occlusions. We propose RAFT…
Scene Flow EstimationSelf-SuperFlow: Self-supervised Scene Flow Prediction in Stereo Sequences
In recent years, deep neural networks showed their exceeding capabilities in addressing many computer vision tasks including scene flow prediction. However, most of the advances are dependent on the availability of a vas…
PredictionAdversarial Self-Supervised Scene Flow Estimation
This work proposes a metric learning approach for self-supervised scene flow estimation. Scene flow estimation is the task of estimating 3D flow vectors for consecutive 3D point clouds. Such flow vectors are fruitful, \e…
Metric LearningScene Flow EstimationSelf-supervised Scene Flow EstimationTripletSelf-Supervised Monocular Scene Flow Estimation
Scene flow estimation has been receiving increasing attention for 3D environment perception. Monocular scene flow estimation -- obtaining 3D structure and 3D motion from two temporally consecutive images -- is a highly i…
Depth EstimationMonocular Depth EstimationOptical Flow EstimationScene Flow Estimation+1