Vision meets mmWave Radar: 3D Object Perception Benchmark for Autonomous Driving
Sensor fusion is crucial for an accurate and robust perception system on autonomous vehicles. Most existing datasets and perception solutions focus on fusing cameras and LiDAR. However, the collaboration between camera and radar is significantly under-exploited. The incorporation of rich semantic information from the camera, and reliable 3D information from the radar can potentially achieve an efficient, cheap, and portable solution for 3D object perception tasks. It can also be robust to different lighting or all-weather driving scenarios due to the capability of mmWave radars. In this paper, we introduce the CRUW3D dataset, including 66K synchronized and well-calibrated camera, radar, and LiDAR frames in various driving scenarios. Unlike other large-scale autonomous driving datasets, our radar data is in the format of radio frequency (RF) tensors that contain not only 3D location information but also spatio-temporal semantic information. This kind of radar format can enable machine learning models to generate more reliable object perception results after interacting and fusing the information or features between the camera and radar.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingAutonomous VehiclesSensor FusionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Sim2Radar: Toward Bridging the Radar Sim-to-Real Gap with VLM-Guided Scene Reconstruction
Millimeter-wave (mmWave) radar provides reliable perception in visually degraded indoor environments (e.g., smoke, dust, and low light), but learning-based radar perception is bottlenecked by the scarcity and cost of col…
Monocular Depth EstimationTransfer LearningObject DetectionRadarTwin: Scene-Specific mmWave Radar Simulation and Learning for Mobile Indoor Perception
Millimeter-wave (mmWave) radar perception is limited by data scarcity: models trained on existing radar datasets fail to generalize to new objects, environments, and sensing trajectories. We present RadarTwin, a framewor…
3D ReconstructionDiffusion-Based Point Cloud Super-Resolution for mmWave Radar Data
The millimeter-wave radar sensor maintains stable performance under adverse environmental conditions, making it a promising solution for all-weather perception tasks, such as outdoor mobile robotics. However, the radar p…
Point Cloud Super ResolutionSuper-ResolutionCan Language Models Understand mmWave Data? Benchmarking Large Language Models for mmWave Radar-Based Human Understanding
Large language models (LLMs) have shown remarkable reasoning and generative capabilities, motivating their use as universal reasoning engines for perception. While modern approaches such as vision-language models (VLMs) …
Question AnsweringRadarXFormer: Robust Object Detection via Cross-Dimension Fusion of 4D Radar Spectra and Images for Autonomous Driving
Reliable perception is essential for autonomous driving systems to operate safely under diverse real-world traffic conditions. However, camera- and LiDAR-based perception systems suffer from performance degradation under…
Robust Object Detection3D Object DetectionAutonomous DrivingPoint Clouds