Optical Flow Estimation for Spiking Camera
As a bio-inspired sensor with high temporal resolution, the spiking camera has an enormous potential in real applications, especially for motion estimation in high-speed scenes. However, frame-based and event-based methods are not well suited to spike streams from the spiking camera due to the different data modalities. To this end, we present, SCFlow, a tailored deep learning pipeline to estimate optical flow in high-speed scenes from spike streams. Importantly, a novel input representation is introduced which can adaptively remove the motion blur in spike streams according to the prior motion. Further, for training SCFlow, we synthesize two sets of optical flow data for the spiking camera, SPIkingly Flying Things and Photo-realistic High-speed Motion, denoted as SPIFT and PHM respectively, corresponding to random high-speed and well-designed scenes. Experimental results show that the SCFlow can predict optical flow from spike streams in different high-speed scenes. Moreover, SCFlow shows promising generalization on \textbf{real spike streams}. Codes and datasets refer to https://github.com/Acnext/Optical-Flow-For-Spiking-Camera.
Code (1)
Tasks
Event-based visionMotion EstimationOptical Flow EstimationSimilar Papers 제목 키워드 기반
SDformerFlow: Spatiotemporal swin spikeformer for event-based optical flow estimation
Event cameras generate asynchronous and sparse event streams capturing changes in light intensity. They offer significant advantages over conventional frame-based cameras, such as a higher dynamic range and an extremely …
Event-based Optical FlowOptical Flow EstimationOptical flow estimation from event-based cameras and spiking neural networks
Event-based cameras are raising interest within the computer vision community. These sensors operate with asynchronous pixels, emitting events, or "spikes", when the luminance change at a given pixel since the last event…
Optical Flow EstimationFusion-FlowNet: Energy-Efficient Optical Flow Estimation using Sensor Fusion and Deep Fused Spiking-Analog Network Architectures
Standard frame-based cameras that sample light intensity frames are heavily impacted by motion blur for high-speed motion and fail to perceive scene accurately when the dynamic range is high. Event-based cameras, on the …
Optical Flow EstimationSensor FusionNeuromorphic Optical Flow and Real-time Implementation with Event Cameras
Optical flow provides information on relative motion that is an important component in many computer vision pipelines. Neural networks provide high accuracy optical flow, yet their complexity is often prohibitive for app…
Event-based visionOptical Flow EstimationEvent-based Temporally Dense Optical Flow Estimation with Sequential Learning
Event cameras provide an advantage over traditional frame-based cameras when capturing fast-moving objects without a motion blur. They achieve this by recording changes in light intensity (known as events), thus allowing…
Event-based Optical FlowOptical Flow Estimation