BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimation
Event cameras provide high temporal precision, low data rates, and high dynamic range visual perception, which are well-suited for optical flow estimation. While data-driven optical flow estimation has obtained great success in RGB cameras, its generalization performance is seriously hindered in event cameras mainly due to the limited and biased training data. In this paper, we present a novel simulator, BlinkSim, for the fast generation of large-scale data for event-based optical flow. BlinkSim incorporates a configurable rendering engine alongside an event simulation suite. By leveraging the wealth of current 3D assets, the rendering engine enables us to automatically build up thousands of scenes with different objects, textures, and motion patterns and render very high-frequency images for realistic event data simulation. Based on BlinkSim, we construct a large training dataset and evaluation benchmark BlinkFlow that contains sufficient, diversiform, and challenging event data with optical flow ground truth. Experiments show that BlinkFlow improves the generalization performance of state-of-the-art methods by more than 40\% on average and up to 90\%. Moreover, we further propose an Event-based optical Flow transFormer (E-FlowFormer) architecture. Powered by our BlinkFlow, E-FlowFormer outperforms the SOTA methods by up to 91\% on the MVSEC dataset and 14\% on the DSEC dataset and presents the best generalization performance. The source code and data are available at https://zju3dv.github.io/blinkflow/.
Code (0)
등록된 구현이 없습니다.
Tasks
Event-based Optical FlowOptical Flow EstimationSimilar Papers 제목 키워드 기반
A Cloud-based Deep Learning Framework for Early Detection of Pushing at Crowded Event Entrances
Crowding at the entrances of large events may lead to critical and life-threatening situations, particularly when people start pushing each other to reach the event faster. Automatic and timely identification of pushing …
Optical Flow EstimationTime Lens: Event-Based Video Frame Interpolation
State-of-the-art frame interpolation methods generate intermediate frames by inferring object motions in the image from consecutive key-frames. In the absence of additional information, first-order approximations, i.…
Optical Flow EstimationVideo Frame InterpolationTimeLens: Event-based Video Frame Interpolation
State-of-the-art frame interpolation methods generate intermediate frames by inferring object motions in the image from consecutive key-frames. In the absence of additional information, first-order approximations, i.e. o…
Optical Flow EstimationVideo Frame InterpolationNeuromorphic Cameras in Astronomy: Unveiling the Future of Celestial Imaging Beyond Conventional Limits
To deepen our understanding of optical astronomy, we must advance imaging technology to overcome conventional frame-based cameras' limited dynamic range and temporal resolution. Our Perspective paper examines how neuromo…
AstronomyBAT: Learning Event-based Optical Flow with Bidirectional Adaptive Temporal Correlation
Event cameras deliver visual information characterized by a high dynamic range and high temporal resolution, offering significant advantages in estimating optical flow for complex lighting conditions and fast-moving obje…
Event-based Optical FlowOptical Flow Estimation