Multi-Modal Depth Estimation Using Convolutional Neural Networks

2020-12-17 · Sadique Adnan Siddiqui, Axel Vierling, Karsten Berns

This paper addresses the problem of dense depth predictions from sparse distance sensor data and a single camera image on challenging weather conditions. This work explores the significance of different sensor modalities such as camera, Radar, and Lidar for estimating depth by applying Deep Learning approaches. Although Lidar has higher depth-sensing abilities than Radar and has been integrated with camera images in lots of previous works, depth estimation using CNN's on the fusion of robust Radar distance data and camera images has not been explored much. In this work, a deep regression network is proposed utilizing a transfer learning approach consisting of an encoder where a high performing pre-trained model has been used to initialize it for extracting dense features and a decoder for upsampling and predicting desired depth. The results are demonstrated on Nuscenes, KITTI, and a Synthetic dataset which was created using the CARLA simulator. Also, top-view zoom-camera images captured from the crane on a construction site are evaluated to estimate the distance of the crane boom carrying heavy loads from the ground to show the usability in safety-critical applications.

📄 PDF Abstract BibTeX arXiv:2012.09667

Code (0)

등록된 구현이 없습니다.

Tasks

Decoder Depth Estimation Transfer Learning

Methods 이 논문이 사용한 방법론

Entropy Regularization 설명 없음

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

CARLA CARLA is an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban…

Demo-Pose: Depth-Monocular Modality Fusion For Object Pose Estimation

2026-03-29 · Rachit Agarwal, Abhishek Joshi, Sathish Chalasani, Woo Jin Kim arxiv

Object pose estimation is a fundamental task in 3D vision with applications in robotics, AR/VR, and scene understanding. We address the challenge of category-level 9-DoF pose estimation (6D pose + 3Dsize) from RGB-D inpu…

Scene Understanding 3D Pose Estimation

PR-GCN: A Deep Graph Convolutional Network with Point Refinement for 6D Pose Estimation

2021-08-23 · ICCV 2021 10 · Guangyuan Zhou, Huiqun Wang, Jiaxin Chen, Di Huang

RGB-D based 6D pose estimation has recently achieved remarkable progress, but still suffers from two major limitations: (1) ineffective representation of depth data and (2) insufficient integration of different modalitie…

6D Pose Estimation Pose Estimation

Analyzing Modular CNN Architectures for Joint Depth Prediction and Semantic Segmentation

2017-02-26 · Omid Hosseini Jafari, Oliver Groth, Alexander Kirillov, Michael Ying Yang 외

This paper addresses the task of designing a modular neural network architecture that jointly solves different tasks. As an example we use the tasks of depth estimation and semantic segmentation given a single RGB image.…

Depth Estimation Depth Prediction Semantic Segmentation

USAM-Net: A U-Net-based Network for Improved Stereo Correspondence and Scene Depth Estimation using Features from a Pre-trained Image Segmentation network

2025-03-19 · Joseph Emmanuel DL Dayo, Prospero C. Naval Jr

The increasing demand for high-accuracy depth estimation in autonomous driving and augmented reality applications necessitates advanced neural architectures capable of effectively leveraging multiple data modalities. In …

Autonomous Driving Depth Estimation Image Segmentation Segmentation+2

Towards Multimodal Depth Estimation from Light Fields

2022-03-30 · CVPR 2022 1 · Titus Leistner, Radek Mackowiak, Lynton Ardizzone, Ullrich Köthe 외

Light field applications, especially light field rendering and depth estimation, developed rapidly in recent years. While state-of-the-art light field rendering methods handle semi-transparent and reflective objects well…

Depth Estimation Depth Prediction