LiDAR-Event Stereo Fusion with Hallucinations
Event stereo matching is an emerging technique to estimate depth from neuromorphic cameras; however, events are unlikely to trigger in the absence of motion or the presence of large, untextured regions, making the correspondence problem extremely challenging. Purposely, we propose integrating a stereo event camera with a fixed-frequency active sensor -- e.g., a LiDAR -- collecting sparse depth measurements, overcoming the aforementioned limitations. Such depth hints are used by hallucinating -- i.e., inserting fictitious events -- the stacks or raw input streams, compensating for the lack of information in the absence of brightness changes. Our techniques are general, can be adapted to any structured representation to stack events and outperform state-of-the-art fusion methods applied to event-based stereo.
Code (1)
Tasks
Stereo MatchingSimilar Papers 제목 키워드 기반
Noise-Aware Unsupervised Deep Lidar-Stereo Fusion
In this paper, we present LidarStereoNet, the first unsupervised Lidar-stereo fusion network, which can be trained in an end-to-end manner without the need of ground truth depth maps. By introducing a novel "Feedback Loo…
3D geometryDepth CompletionStereo MatchingStereo Matching Hand3D LiDAR and Stereo Fusion using Stereo Matching Network with Conditional Cost Volume Normalization
The complementary characteristics of active and passive depth sensing techniques motivate the fusion of the Li-DAR sensor and stereo camera for improved depth perception. Instead of directly fusing estimated depths acros…
Depth CompletionStereo-LiDAR FusionStereo MatchingStereo Matching HandVolumetric Propagation Network: Stereo-LiDAR Fusion for Long-Range Depth Estimation
Stereo-LiDAR fusion is a promising task in that we can utilize two different types of 3D perceptions for practical usage -- dense 3D information (stereo cameras) and highly-accurate sparse point clouds (LiDAR). However, …
Depth CompletionDepth EstimationSensor FusionStereo-LiDAR Fusion+2DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDAR
In this paper, we present DSERT-RoLL, a driving dataset that incorporates stereo event, RGB, and thermal cameras together with 4D radar and dual LiDAR, collected across diverse weather and illumination conditions. The da…
Frustum Fusion: Pseudo-LiDAR and LiDAR Fusion for 3D Detection
Most autonomous vehicles are equipped with LiDAR sensors and stereo cameras. The former is very accurate but generates sparse data, whereas the latter is dense, has rich texture and color information but difficult to ext…
3D Object DetectionAutonomous Vehiclesobject-detectionObject Detection