RC-BEVFusion: A Plug-In Module for Radar-Camera Bird's Eye View Feature Fusion
Radars and cameras belong to the most frequently used sensors for advanced driver assistance systems and automated driving research. However, there has been surprisingly little research on radar-camera fusion with neural networks. One of the reasons is a lack of large-scale automotive datasets with radar and unmasked camera data, with the exception of the nuScenes dataset. Another reason is the difficulty of effectively fusing the sparse radar point cloud on the bird's eye view (BEV) plane with the dense images on the perspective plane. The recent trend of camera-based 3D object detection using BEV features has enabled a new type of fusion, which is better suited for radars. In this work, we present RC-BEVFusion, a modular radar-camera fusion network on the BEV plane. We propose BEVFeatureNet, a novel radar encoder branch, and show that it can be incorporated into several state-of-the-art camera-based architectures. We show significant performance gains of up to 28% increase in the nuScenes detection score, which is an important step in radar-camera fusion research. Without tuning our model for the nuScenes benchmark, we achieve the best result among all published methods in the radar-camera fusion category.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object Detectionobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
UniBEVFusion: Unified Radar-Vision BEVFusion for 3D Object Detection
4D millimeter-wave (MMW) radar, which provides both height information and dense point cloud data over 3D MMW radar, has become increasingly popular in 3D object detection. In recent years, radar-vision fusion models hav…
3D Object DetectionDepth EstimationDepth Predictionobject-detection+1CoBEVFusion: Cooperative Perception with LiDAR-Camera Bird's-Eye View Fusion
Autonomous Vehicles (AVs) use multiple sensors to gather information about their surroundings. By sharing sensor data between Connected Autonomous Vehicles (CAVs), the safety and reliability of these vehicles can be impr…
3D Object DetectionAutonomous Vehiclesobject-detectionObject Detection+1BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidance and Temporal Aggregation
Integrating LiDAR and Camera information into Bird's-Eye-View (BEV) has become an essential topic for 3D object detection in autonomous driving. Existing methods mostly adopt an independent dual-branch framework to gener…
3D Object DetectionAutonomous Drivingobject-detectionObject DetectionBEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation
Multi-sensor fusion is essential for an accurate and reliable autonomous driving system. Recent approaches are based on point-level fusion: augmenting the LiDAR point cloud with camera features. However, the camera-to-Li…
3D Multi-Object Tracking3D Object DetectionAutonomous DrivingObject Detection+2SB-BEVFusion: Enhancing the Robustness against Sensor Malfunction and Corruptions
Multimodal sensor fusion has demonstrated remarkable performance improvements over unimodal approaches in 3D object detection for autonomous vehicles. Typically, existing methods transform multimodal data from independen…
Autonomous Vehicles3D Object Detection