Multimodal Point Cloud Semantic Segmentation With Virtual Point Enhancement

2025-04-02 · Zaipeng Duan, Xuzhong Hu, Pei An, Jie Ma

LiDAR-based 3D point cloud recognition has been proven beneficial in various applications. However, the sparsity and varying density pose a significant challenge in capturing intricate details of objects, particularly for medium-range and small targets. Therefore, we propose a multi-modal point cloud semantic segmentation method based on Virtual Point Enhancement (VPE), which integrates virtual points generated from images to address these issues. These virtual points are dense but noisy, and directly incorporating them can increase computational burden and degrade performance. Therefore, we introduce a spatial difference-driven adaptive filtering module that selectively extracts valuable pseudo points from these virtual points based on density and distance, enhancing the density of medium-range targets. Subsequently, we propose a noise-robust sparse feature encoder that incorporates noise-robust feature extraction and fine-grained feature enhancement. Noise-robust feature extraction exploits the 2D image space to reduce the impact of noisy points, while fine-grained feature enhancement boosts sparse geometric features through inner-voxel neighborhood point aggregation and downsampled voxel aggregation. The results on the SemanticKITTI and nuScenes, two large-scale benchmark data sets, have validated effectiveness, significantly improving 2.89\% mIoU with the introduction of 7.7\% virtual points on nuScenes.

📄 PDF Abstract BibTeX arXiv:2504.01449

Code (0)

등록된 구현이 없습니다.

Tasks

Semantic Segmentation

VirtualPainting: Addressing Sparsity with Virtual Points and Distance-Aware Data Augmentation for 3D Object Detection

2023-12-26 · Sudip Dhakal, Dominic Carrillo, Deyuan Qu, Michael Nutt 외

In recent times, there has been a notable surge in multimodal approaches that decorates raw LiDAR point clouds with camera-derived features to improve object detection performance. However, we found that these methods st…

2D Semantic Segmentation 3D Object Detection Data Augmentation object-detection+2

MNEW: Multi-domain Neighborhood Embedding and Weighting for Sparse Point Clouds Segmentation

2020-04-05 · Yang Zheng, Izzat H. Izzat, Sanling Song

Point clouds have been widely adopted in 3D semantic scene understanding. However, point clouds for typical tasks such as 3D shape segmentation or indoor scenario parsing are much denser than outdoor LiDAR sweeps for the…

Autonomous Driving Scene Understanding Semantic Segmentation

Multi-Resolution Graph Neural Network for Large-Scale Pointcloud Segmentation

2020-09-18 · Liuyue Xie, Tomotake Furuhata, Kenji Shimada

In this paper, we propose a multi-resolution deep-learning architecture to semantically segment dense large-scale pointclouds. Dense pointcloud data require a computationally expensive feature encoding process before sem…

GPU Graph Neural Network Segmentation Semantic Segmentation

MuGNet: Multi-Resolution Graph Neural Network for Large-Scale Pointcloud Segmentation

2020-11-16 · Conference on Robot Learning 2020 11 · Liuyue Xie, Tomotake Furuhata, Kenji Shimada

GPU Graph Neural Network Segmentation Semantic Segmentation

MVCTrack: Boosting 3D Point Cloud Tracking via Multimodal-Guided Virtual Cues

2024-12-03 · Zhaofeng Hu, Sifan Zhou, Shibo Zhao, Zhihang Yuan 외

3D single object tracking is essential in autonomous driving and robotics. Existing methods often struggle with sparse and incomplete point cloud scenarios. To address these limitations, we propose a Multimodal-guided Vi…

3D Single Object Tracking Autonomous Driving Object Tracking