Optical flow and scene flow estimation: A survey
Motion analysis is one of the most fundamental and challenging problems in the field of computer vision, which can be widely applied in many areas, such as autonomous driving, action recognition, scene understanding, and robotics. In general, the displacement field between subsequent frames can be divided into two types: optical flow and scene flow. The optical flow represents the pixel motion of adjacent frames. In contrast, the scene flow is a 3D motion field of the dynamic scene between two frames. Traditional approaches for the estimation of optical flow and scene flow usually leverage the variational technique, which can be solved as an energy minimization process. In recent years, deep learning has emerged as a powerful technique for learning feature representations directly from data. It has led to remarkable progress in the field of optical flow and scene flow estimation. In this paper, we provide a comprehensive survey of optical flow and scene flow estimation. First, we briefly review the pioneering approaches that use variational technique and then we delve in detail into the deep learning-based approaches. Furthermore, we present insightful observations on evaluation issues, specifically benchmark datasets, evaluation metrics, and state-of-the-art performance. Finally, we give the promising directions for future research. To the best of our knowledge, we are the first to review both optical flow and scene flow estimation, and the first to cover both traditional and deep learning-based approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionAutonomous DrivingDeep LearningOptical Flow EstimationScene Flow EstimationScene UnderstandingSurveySimilar Papers 제목 키워드 기반
Upgrading Optical Flow to 3D Scene Flow Through Optical Expansion
We describe an approach for upgrading 2D optical flow to 3D scene flow. Our key insight is that dense optical expansion - which can be reliably inferred from monocular frame pairs - reveals changes in depth of scene elem…
Depth EstimationOptical Flow EstimationA Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation
Recent work has shown that optical flow estimation can be formulated as a supervised learning task and can be successfully solved with convolutional networks. Training of the so-called FlowNet was enabled by a large synt…
Disparity EstimationOptical Flow EstimationScene Flow EstimationCombining Stereo Disparity and Optical Flow for Basic Scene Flow
Scene flow is a description of real world motion in 3D that contains more information than optical flow. Because of its complexity there exists no applicable variant for real-time scene flow estimation in an automotive o…
Optical Flow EstimationScene Flow EstimationSSTM: Spatiotemporal Recurrent Transformers for Multi-frame Optical Flow Estimation
Inaccurate optical flow estimates in and near occluded regions, and out-of-boundary regions are two of the current significant limitations of optical flow estimation algorithms. Recent state-of-the-art optical flow estim…
Optical Flow EstimationDense Scene Flow from Stereo Disparity and Optical Flow
Scene flow describes 3D motion in a 3D scene. It can either be modeled as a single task, or it can be reconstructed from the auxiliary tasks of stereo depth and optical flow estimation. While the second method can achiev…
Optical Flow EstimationScene Flow Estimation