Self-supervised Event-based Monocular Depth Estimation using Cross-modal Consistency
An event camera is a novel vision sensor that can capture per-pixel brightness changes and output a stream of asynchronous ``events''. It has advantages over conventional cameras in those scenes with high-speed motions and challenging lighting conditions because of the high temporal resolution, high dynamic range, low bandwidth, low power consumption, and no motion blur. Therefore, several supervised monocular depth estimation from events is proposed to address scenes difficult for conventional cameras. However, depth annotation is costly and time-consuming. In this paper, to lower the annotation cost, we propose a self-supervised event-based monocular depth estimation framework named EMoDepth. EMoDepth constrains the training process using the cross-modal consistency from intensity frames that are aligned with events in the pixel coordinate. Moreover, in inference, only events are used for monocular depth prediction. Additionally, we design a multi-scale skip-connection architecture to effectively fuse features for depth estimation while maintaining high inference speed. Experiments on MVSEC and DSEC datasets demonstrate that our contributions are effective and that the accuracy can outperform existing supervised event-based and unsupervised frame-based methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Depth EstimationDepth PredictionMonocular Depth EstimationSimilar Papers 제목 키워드 기반
Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance
Self-supervised monocular depth estimation presents a powerful method to obtain 3D scene information from single camera images, which is trainable on arbitrary image sequences without requiring depth labels, e.g., from a…
Depth EstimationMonocular Depth EstimationSemantic SegmentationSelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learning
Monocular depth estimation in the wild inherently predicts depth up to an unknown scale. To resolve scale ambiguity issue, we present a learning algorithm that leverages monocular simultaneous localization and mapping (S…
Depth EstimationMonocular Depth EstimationRobot NavigationSelf-Supervised Learning+1RealMonoDepth: Self-Supervised Monocular Depth Estimation for General Scenes
We present a generalised self-supervised learning approach for monocular estimation of the real depth across scenes with diverse depth ranges from 1--100s of meters. Existing supervised methods for monocular depth estima…
Depth EstimationMonocular Depth EstimationSelf-Supervised LearningFusionDepth: Complement Self-Supervised Monocular Depth Estimation with Cost Volume
Multi-view stereo depth estimation based on cost volume usually works better than self-supervised monocular depth estimation except for moving objects and low-textured surfaces. So in this paper, we propose a multi-frame…
Depth EstimationMonocular Depth EstimationStereo Depth EstimationA high-precision self-supervised monocular visual odometry in foggy weather based on robust cycled generative adversarial networks and multi-task learning aided depth estimation
This paper proposes a high-precision self-supervised monocular VO, which is specifically designed for navigation in foggy weather. A cycled generative adversarial network is designed to obtain high-quality self-supervise…
Depth EstimationGenerative Adversarial NetworkMonocular Visual OdometryMulti-Task Learning+2