Joint 3D Shape and Motion Estimation from Rolling Shutter Light-Field Images
In this paper, we propose an approach to address the problem of 3D reconstruction of scenes from a single image captured by a light-field camera equipped with a rolling shutter sensor. Our method leverages the 3D information cues present in the light-field and the motion information provided by the rolling shutter effect. We present a generic model for the imaging process of this sensor and a two-stage algorithm that minimizes the re-projection error while considering the position and motion of the camera in a motion-shape bundle adjustment estimation strategy. Thereby, we provide an instantaneous 3D shape-and-pose-and-velocity sensing paradigm. To the best of our knowledge, this is the first study to leverage this type of sensor for this purpose. We also present a new benchmark dataset composed of different light-fields showing rolling shutter effects, which can be used as a common base to improve the evaluation and tracking the progress in the field. We demonstrate the effectiveness and advantages of our approach through several experiments conducted for different scenes and types of motions. The source code and dataset are publicly available at: https://github.com/ICB-Vision-AI/RSLF
Code (1)
Tasks
3D ReconstructionMotion EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Inertial-aided Rolling Shutter Relative Pose Estimation
Relative pose estimation is a fundamental problem in computer vision and it has been studied for conventional global shutter cameras for decades. However, recently, a rolling shutter camera has been widely used due to it…
Pose EstimationRolling Shutter and Radial Distortion Are Features for High Frame Rate Multi-Camera Tracking
Traditionally, camera-based tracking approaches have treated rolling shutter and radial distortion as imaging artifacts that have to be overcome and corrected for in order to apply standard camera models and scene recons…
Camera Pose EstimationMotion EstimationPose EstimationGeometric Models of Rolling-Shutter Cameras
Cameras with rolling shutters are becoming more common as low-power, low-cost CMOS sensors are being used more frequently in cameras. The rolling shutter means that not all scanlines are exposed over the same time interv…
Rolling Shutter Pose and Ego-motion Estimation using Shape-from-Template
We propose a new method for the absolute camera pose problem (PnP) which handles Rolling Shutter (RS) effects. Unlike all existing methods which perform 3D-2D registration after augmenting the Global Shutter (GS) project…
Motion EstimationPose EstimationGyroscope-aided Relative Pose Estimation for Rolling Shutter Cameras
The rolling shutter camera has received great attention due to its low cost imaging capability, however, the estimation of relative pose between rolling shutter cameras still remains a difficult problem owing to its line…
Pose Estimation