Unsupervised Learning of Dense Optical Flow, Depth and Egomotion from Sparse Event Data
In this work we present a lightweight, unsupervised learning pipeline for \textit{dense} depth, optical flow and egomotion estimation from sparse event output of the Dynamic Vision Sensor (DVS). To tackle this low level vision task, we use a novel encoder-decoder neural network architecture - ECN. Our work is the first monocular pipeline that generates dense depth and optical flow from sparse event data only. The network works in self-supervised mode and has just 150k parameters. We evaluate our pipeline on the MVSEC self driving dataset and present results for depth, optical flow and and egomotion estimation. Due to the lightweight design, the inference part of the network runs at 250 FPS on a single GPU, making the pipeline ready for realtime robotics applications. Our experiments demonstrate significant improvements upon previous works that used deep learning on event data, as well as the ability of our pipeline to perform well during both day and night.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderGPUOptical Flow EstimationSimilar Papers 제목 키워드 기반
E-MoFlow: Learning Egomotion and Optical Flow from Event Data via Implicit Regularization
The estimation of optical flow and 6-DoF ego-motion, two fundamental tasks in 3D vision, has typically been addressed independently. For neuromorphic vision (e.g., event cameras), however, the lack of robust data associa…
Optical Flow EstimationDepth EstimationUnsupervised Event-based Learning of Optical Flow, Depth, and Egomotion
In this work, we propose a novel framework for unsupervised learning for event cameras that learns motion information from only the event stream. In particular, we propose an input representation of the events in the for…
Optical Flow EstimationUnsupervised Learning of Depth, Camera Pose and Optical Flow from Monocular Video
We propose DFPNet -- an unsupervised, joint learning system for monocular Depth, Optical Flow and egomotion (Camera Pose) estimation from monocular image sequences. Due to the nature of 3D scene geometry these three comp…
Camera Pose EstimationImage ReconstructionOptical Flow EstimationPose EstimationJoint Unsupervised Learning of Optical Flow and Egomotion with Bi-Level Optimization
We address the problem of joint optical flow and camera motion estimation in rigid scenes by incorporating geometric constraints into an unsupervised deep learning framework. Unlike existing approaches which rely on brig…
Motion EstimationOptical Flow EstimationMotion Segmentation and Egomotion Estimation from Event-Based Normal Flow
This paper introduces a robust framework for motion segmentation and egomotion estimation using event-based normal flow, tailored specifically for neuromorphic vision sensors. In contrast to traditional methods that rely…
Motion SegmentationDepth Estimation