Radar Spectra-Language Model for Automotive Scene Parsing
Radar sensors are low cost, long-range, and weather-resilient. Therefore, they are widely used for driver assistance functions, and are expected to be crucial for the success of autonomous driving in the future. In many perception tasks only pre-processed radar point clouds are considered. In contrast, radar spectra are a raw form of radar measurements and contain more information than radar point clouds. However, radar spectra are rather difficult to interpret. In this work, we aim to explore the semantic information contained in spectra in the context of automated driving, thereby moving towards better interpretability of radar spectra. To this end, we create a radar spectra-language model, allowing us to query radar spectra measurements for the presence of scene elements using free text. We overcome the scarcity of radar spectra data by matching the embedding space of an existing vision-language model. Finally, we explore the benefit of the learned representation for scene retrieval using radar spectra only, and obtain improvements in free space segmentation and object detection merely by injecting the spectra embedding into a baseline model.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous DrivingLanguage ModelingLanguage Modellingmodelobject-detectionObject DetectionScene ParsingSimilar Papers 제목 키워드 기반
Warping of Radar Data into Camera Image for Cross-Modal Supervision in Automotive Applications
We present an approach to automatically generate semantic labels for real recordings of automotive range-Doppler (RD) radar spectra. Such labels are required when training a neural network for object recognition from rad…
Direction of Arrival EstimationObject RecognitionScene Flow EstimationSemantic SegmentationRedefining Automotive Radar Imaging: A Domain-Informed 1D Deep Learning Approach for High-Resolution and Efficient Performance
Millimeter-wave (mmWave) radars are indispensable for perception tasks of autonomous vehicles, thanks to their resilience in challenging weather conditions. Yet, their deployment is often limited by insufficient spatial …
Autonomous VehiclesSuper-ResolutionDAROD: A Deep Automotive Radar Object Detector on Range-Doppler maps
Due to the small number of raw data automotive radar datasets and the low resolution of such radar sensors, automotive radar object detection has been little explored with deep learning models in comparison to camera and…
2D Object DetectionObjectobject-detectionObject Detection+1Self-Supervised Scene Flow Estimation with 4-D Automotive Radar
Scene flow allows autonomous vehicles to reason about the arbitrary motion of multiple independent objects which is the key to long-term mobile autonomy. While estimating the scene flow from LiDAR has progressed recently…
Autonomous VehiclesMotion SegmentationScene Flow EstimationSelf-Supervised Learning+1DeepHybrid: Deep Learning on Automotive Radar Spectra and Reflections for Object Classification
Automated vehicles need to detect and classify objects and traffic participants accurately. Reliable object classification using automotive radar sensors has proved to be challenging. We propose a method that combines cl…
Collision AvoidanceNeural Architecture Search