Hallucinating Dense Optical Flow from Sparse Lidar for Autonomous Vehicles
In this paper we propose a novel approach to estimate dense optical flow from sparse lidar data acquired on an autonomous vehicle. This is intended to be used as a drop-in replacement of any image-based optical flow system when images are not reliable due to e.g. adverse weather conditions or at night. In order to infer high resolution 2D flows from discrete range data we devise a three-block architecture of multiscale filters that combines multiple intermediate objectives, both in the lidar and image domain. To train this network we introduce a dataset with approximately 20K lidar samples of the Kitti dataset which we have augmented with a pseudo ground-truth image-based optical flow computed using FlowNet2. We demonstrate the effectiveness of our approach on Kitti, and show that despite using the low-resolution and sparse measurements of the lidar, we can regress dense optical flow maps which are at par with those estimated with image-based methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous VehiclesOptical Flow EstimationSimilar Papers 제목 키워드 기반
FPPN: Future Pseudo-LiDAR Frame Prediction for Autonomous Driving
LiDAR sensors are widely used in autonomous driving due to the reliable 3D spatial information. However, the data of LiDAR is sparse and the frequency of LiDAR is lower than that of cameras. To generate denser point clou…
Autonomous DrivingOptical Flow EstimationPredictionDeGraF-Flow: Extending DeGraF Features for accurate and efficient sparse-to-dense optical flow estimation
Modern optical flow methods make use of salient scene feature points detected and matched within the scene as a basis for sparse-to-dense optical flow estimation. Current feature detectors however either give sparse, non…
Computational EfficiencyGPUOptical Flow EstimationUpgrading Optical Flow to 3D Scene Flow Through Optical Expansion
We describe an approach for upgrading 2D optical flow to 3D scene flow. Our key insight is that dense optical expansion - which can be reliably inferred from monocular frame pairs - reveals changes in depth of scene elem…
Depth EstimationOptical Flow EstimationEfficient Sparse-to-Dense Optical Flow Estimation Using a Learned Basis and Layers
We address the elusive goal of estimating optical flow both accurately and efficiently by adopting a sparse-to-dense approach. Given a set of sparse matches, we regress to dense optical flow using a learned set of full-f…
CPUOptical Flow EstimationLiDAR-Flow: Dense Scene Flow Estimation from Sparse LiDAR and Stereo Images
We propose a new approach called LiDAR-Flow to robustly estimate a dense scene flow by fusing a sparse LiDAR with stereo images. We take the advantage of the high accuracy of LiDAR to resolve the lack of information in s…
Scene Flow Estimation