A Versatile Multi-View Framework for LiDAR-based 3D Object Detection with Guidance from Panoptic Segmentation
3D object detection using LiDAR data is an indispensable component for autonomous driving systems. Yet, only a few LiDAR-based 3D object detection methods leverage segmentation information to further guide the detection process. In this paper, we propose a novel multi-task framework that jointly performs 3D object detection and panoptic segmentation. In our method, the 3D object detection backbone in Bird's-Eye-View (BEV) plane is augmented by the injection of Range-View (RV) feature maps from the 3D panoptic segmentation backbone. This enables the detection backbone to leverage multi-view information to address the shortcomings of each projection view. Furthermore, foreground semantic information is incorporated to ease the detection task by highlighting the locations of each object class in the feature maps. Finally, a new center density heatmap generated based on the instance-level information further guides the detection backbone by suggesting possible box center locations for objects. Our method works with any BEV-based 3D object detection method, and as shown by extensive experiments on the nuScenes dataset, it provides significant performance gains. Notably, the proposed method based on a single-stage CenterPoint 3D object detection network achieved state-of-the-art performance on nuScenes 3D Detection Benchmark with 67.3 NDS.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object Detection3D Panoptic SegmentationAutonomous DrivingObjectobject-detectionObject DetectionPanoptic SegmentationSegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Small, Versatile and Mighty: A Range-View Perception Framework
Despite its compactness and information integrity, the range view representation of LiDAR data rarely occurs as the first choice for 3D perception tasks. In this work, we further push the envelop of the range-view repres…
Panoptic SegmentationSemantic SegmentationVALO: A Versatile Anytime Framework for LiDAR-based Object Detection Deep Neural Networks
This work addresses the challenge of adapting dynamic deadline requirements for LiDAR object detection deep neural networks (DNNs). The computing latency of object detection is critically important to ensure safe and eff…
Objectobject-detectionObject DetectionVirtualPainting: Addressing Sparsity with Virtual Points and Distance-Aware Data Augmentation for 3D Object Detection
In recent times, there has been a notable surge in multimodal approaches that decorates raw LiDAR point clouds with camera-derived features to improve object detection performance. However, we found that these methods st…
2D Semantic Segmentation3D Object DetectionData Augmentationobject-detection+2Weak-to-Strong 3D Object Detection with X-Ray Distillation
This paper addresses the critical challenges of sparsity and occlusion in LiDAR-based 3D object detection. Current methods often rely on supplementary modules or specific architectural designs, potentially limiting their…
3D Object DetectionAutonomous DrivingKnowledge DistillationObject+2LiDAR-NeRF: Novel LiDAR View Synthesis via Neural Radiance Fields
We introduce a new task, novel view synthesis for LiDAR sensors. While traditional model-based LiDAR simulators with style-transfer neural networks can be applied to render novel views, they fall short of producing accur…
3D ReconstructionNeRFNovel LiDAR View SynthesisNovel View Synthesis+1