BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation

2022-05-26 · Zhijian Liu, Haotian Tang, Alexander Amini, Xinyu Yang, Huizi Mao, Daniela Rus, Song Han

Multi-sensor fusion is essential for an accurate and reliable autonomous driving system. Recent approaches are based on point-level fusion: augmenting the LiDAR point cloud with camera features. However, the camera-to-LiDAR projection throws away the semantic density of camera features, hindering the effectiveness of such methods, especially for semantic-oriented tasks (such as 3D scene segmentation). In this paper, we break this deeply-rooted convention with BEVFusion, an efficient and generic multi-task multi-sensor fusion framework. It unifies multi-modal features in the shared bird's-eye view (BEV) representation space, which nicely preserves both geometric and semantic information. To achieve this, we diagnose and lift key efficiency bottlenecks in the view transformation with optimized BEV pooling, reducing latency by more than 40x. BEVFusion is fundamentally task-agnostic and seamlessly supports different 3D perception tasks with almost no architectural changes. It establishes the new state of the art on nuScenes, achieving 1.3% higher mAP and NDS on 3D object detection and 13.6% higher mIoU on BEV map segmentation, with 1.9x lower computation cost. Code to reproduce our results is available at https://github.com/mit-han-lab/bevfusion.

📄 PDF Abstract BibTeX arXiv:2205.13542

Code (2)

mit-han-lab/bevfusion 공식 구현 pytorch

nvidia-ai-iot/lidar_ai_solution pytorch

Tasks

3D Multi-Object Tracking 3D Object Detection Autonomous Driving Object Detection Scene SegmentationSensor Fusion

CoBEVFusion: Cooperative Perception with LiDAR-Camera Bird's-Eye View Fusion

2023-10-09 · Donghao Qiao, Farhana Zulkernine

Autonomous Vehicles (AVs) use multiple sensors to gather information about their surroundings. By sharing sensor data between Connected Autonomous Vehicles (CAVs), the safety and reliability of these vehicles can be impr…

3D Object Detection Autonomous Vehicles object-detection Object Detection+1

SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions

2026-05-12 · Markus Essl, Marta Moscati, Mubashir Noman, Muhammad Zaigham Zaheer 외 arxiv

Multimodal sensor fusion has demonstrated remarkable performance improvements over unimodal approaches in 3D object detection for autonomous vehicles. Typically, existing methods transform multimodal data from independen…

Autonomous Vehicles 3D Object Detection

SemanticBEVFusion: Rethink LiDAR-Camera Fusion in Unified Bird's-Eye View Representation for 3D Object Detection

2022-12-09 · Qi Jiang, Hao Sun, Xi Zhang

LiDAR and camera are two essential sensors for 3D object detection in autonomous driving. LiDAR provides accurate and reliable 3D geometry information while the camera provides rich texture with color. Despite the increa…

3D geometry 3D Object Detection Autonomous Driving object-detection+1

Evaluating the Impact of Weather-Induced Sensor Occlusion on BEVFusion for 3D Object Detection

2025-11-06 · Sanjay Kumar, Tim Brophy, Eoin Martino Grua, Ganesh Sistu 외 arxiv

Accurate 3D object detection is essential for automated vehicles to navigate safely in complex real-world environments. Bird's Eye View (BEV) representations, which project multi-sensor data into a top-down spatial forma…

3D Object Detection

BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework

2022-05-27 · TingTing Liang, Hongwei Xie, Kaicheng Yu, Zhongyu Xia 외

Fusing the camera and LiDAR information has become a de-facto standard for 3D object detection tasks. Current methods rely on point clouds from the LiDAR sensor as queries to leverage the feature from the image space. Ho…

3D Object Detection Autonomous Driving object-detection Object Detection