Deep Optics for Monocular Depth Estimation and 3D Object Detection
Depth estimation and 3D object detection are critical for scene understanding but remain challenging to perform with a single image due to the loss of 3D information during image capture. Recent models using deep neural networks have improved monocular depth estimation performance, but there is still difficulty in predicting absolute depth and generalizing outside a standard dataset. Here we introduce the paradigm of deep optics, i.e. end-to-end design of optics and image processing, to the monocular depth estimation problem, using coded defocus blur as an additional depth cue to be decoded by a neural network. We evaluate several optical coding strategies along with an end-to-end optimization scheme for depth estimation on three datasets, including NYU Depth v2 and KITTI. We find an optimized freeform lens design yields the best results, but chromatic aberration from a singlet lens offers significantly improved performance as well. We build a physical prototype and validate that chromatic aberrations improve depth estimation on real-world results. In addition, we train object detection networks on the KITTI dataset and show that the lens optimized for depth estimation also results in improved 3D object detection performance.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object DetectionDepth EstimationMonocular Depth EstimationObjectobject-detectionObject DetectionScene UnderstandingSimilar Papers 제목 키워드 기반
Scratched Lenses, Shifted Depth: Passive Camera-Side Optical Attacks
Physical adversarial attacks on vision systems are typically studied through scene manipulation, such as adversarial patches or projections, where the adversary controls what the camera observes. Camera-side attacks usin…
Monocular 3D Object DetectionMonocular Depth EstimationDepth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking
Monocular image-based 3D perception has become an active research area in recent years owing to its applications in autonomous driving. Approaches to monocular 3D perception including detection and tracking, however, oft…
Autonomous DrivingDepth EstimationObjectTask-Aware Monocular Depth Estimation for 3D Object Detection
Monocular depth estimation enables 3D perception from a single 2D image, thus attracting much research attention for years. Almost all methods treat foreground and background regions ("things and stuff") in an image equa…
3D Object Detection3D Object RecognitionDepth EstimationDepth Prediction+5Learning Geometry-Guided Depth via Projective Modeling for Monocular 3D Object Detection
As a crucial task of autonomous driving, 3D object detection has made great progress in recent years. However, monocular 3D object detection remains a challenging problem due to the unsatisfactory performance in depth es…
3D Object DetectionAutonomous DrivingDepth EstimationMonocular 3D Object Detection+43D Object Aided Self-Supervised Monocular Depth Estimation
Monocular depth estimation has been actively studied in fields such as robot vision, autonomous driving, and 3D scene understanding. Given a sequence of color images, unsupervised learning methods based on the framework …
3D Object DetectionAutonomous DrivingDepth EstimationMonocular 3D Object Detection+5