CVT-Occ: Cost Volume Temporal Fusion for 3D Occupancy Prediction

2024-09-20 · Zhangchen Ye, Tao Jiang, Chenfeng Xu, Yiming Li, Hang Zhao

Vision-based 3D occupancy prediction is significantly challenged by the inherent limitations of monocular vision in depth estimation. This paper introduces CVT-Occ, a novel approach that leverages temporal fusion through the geometric correspondence of voxels over time to improve the accuracy of 3D occupancy predictions. By sampling points along the line of sight of each voxel and integrating the features of these points from historical frames, we construct a cost volume feature map that refines current volume features for improved prediction outcomes. Our method takes advantage of parallax cues from historical observations and employs a data-driven approach to learn the cost volume. We validate the effectiveness of CVT-Occ through rigorous experiments on the Occ3D-Waymo dataset, where it outperforms state-of-the-art methods in 3D occupancy prediction with minimal additional computational cost. The code is released at \url{https://github.com/Tsinghua-MARS-Lab/CVT-Occ}.

📄 PDF Abstract BibTeX arXiv:2409.13430

Code (1)

Tsinghua-MARS-Lab/CVT-Occ 공식 구현 pytorch

Tasks

Depth Estimation Prediction

MetaOcc: Surround-View 4D Radar and Camera Fusion Framework for 3D Occupancy Prediction with Dual Training Strategies

2025-01-26 · Long Yang, Lianqing Zheng, Wenjin Ai, Minghao Liu 외

3D occupancy prediction is crucial for autonomous driving perception. Fusion of 4D radar and camera provides a potential solution of robust occupancy prediction on serve weather with least cost. How to achieve effective …

Autonomous Driving Prediction Pseudo Label

Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Prediction

2024-04-06 · Jingyi Pan, Zipeng Wang, Lin Wang

3D semantic occupancy prediction is a pivotal task in the field of autonomous driving. Recent approaches have made great advances in 3D semantic occupancy predictions on a single modality. However, multi-modal semantic o…

3D Semantic Occupancy Prediction Autonomous Driving Prediction

ALOcc: Adaptive Lifting-based 3D Semantic Occupancy and Cost Volume-based Flow Prediction

2024-11-12 · Dubing Chen, Jin Fang, Wencheng Han, Xinjing Cheng 외

Vision-based semantic occupancy and flow prediction plays a crucial role in providing spatiotemporal cues for real-world tasks, such as autonomous driving. Existing methods prioritize higher accuracy to cater to the dema…

3D Semantic Occupancy Prediction Autonomous Driving Denoising Prediction

Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction

2024-07-10 · Yili Liu, Linzhan Mou, Xuan Yu, Chenrui Han 외

Accurate perception of the dynamic environment is a fundamental task for autonomous driving and robot systems. This paper introduces Let Occ Flow, the first self-supervised work for joint 3D occupancy and occupancy flow …

Autonomous Driving Optical Flow Estimation Prediction

Hierarchical Context Alignment with Disentangled Geometric and Temporal Modeling for Semantic Occupancy Prediction

2024-12-11 · Bohan Li, Xin Jin, Jiajun Deng, Yasheng Sun 외

Camera-based 3D Semantic Occupancy Prediction (SOP) is crucial for understanding complex 3D scenes from limited 2D image observations. Existing SOP methods typically aggregate contextual features to assist the occupancy …

3D Semantic Occupancy Prediction LIDAR Semantic Segmentation Representation Learning Semantic Segmentation