Open-Set Semantic Segmentation for Point Clouds via Adversarial Prototype Framework
Recently, point cloud semantic segmentation has attracted much attention in computer vision. Most of the existing works in literature assume that the training and testing point clouds have the same object classes, but they are generally invalid in many real-world scenarios for identifying the 3D objects whose classes are not seen in the training set. To address this problem, we propose an Adversarial Prototype Framework (APF) for handling the open-set 3D semantic segmentation task, which aims to identify 3D unseen-class points while maintaining the segmentation performance on seen-class points. The proposed APF consists of a feature extraction module for extracting point features, a prototypical constraint module, and a feature adversarial module. The prototypical constraint module is designed to learn prototypes for each seen class from point features. The feature adversarial module utilizes generative adversarial networks to estimate the distribution of unseen-class features implicitly, and the synthetic unseen-class features are utilized to prompt the model to learn more effective point features and prototypes for discriminating unseen-class samples from the seen-class ones. Experimental results on two public datasets demonstrate that the proposed APF outperforms the comparative methods by a large margin in most cases.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Semantic SegmentationSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
JOPP-3D: Joint Open Vocabulary Semantic Segmentation on Point Clouds and Panoramas
Semantic segmentation across visual modalities such as 3D point clouds and panoramic images remains a challenging task, primarily due to the scarcity of annotated data and the limited adaptability of fixed-label models. …
Open Vocabulary Semantic Segmentation3D Semantic SegmentationScene UnderstandingPoint CloudsOpenUrban3D: Annotation-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds
Open-vocabulary semantic segmentation enables models to recognize and segment objects from arbitrary natural language descriptions, offering the flexibility to handle novel, fine-grained, or functionally defined categori…
Point Cloud SegmentationSemantic SegmentationScene UnderstandingPoint CloudsOpen-world Semantic Segmentation for LIDAR Point Clouds
Current methods for LIDAR semantic segmentation are not robust enough for real-world applications, e.g., autonomous driving, since it is closed-set and static. The closed-set assumption makes the network only able to out…
Autonomous DrivingIncremental LearningLIDAR Semantic SegmentationSegmentation+1Exploring Spatial Context for 3D Semantic Segmentation of Point Clouds
Deep learning approaches have made tremendous progress in the field of semantic segmentation over the past few years. However, most current approaches operate in the 2D image space. Direct semantic segmentation of unstru…
3D Semantic SegmentationSegmentationSemantic SegmentationCOSTA: A Cluster-Centric Paradigm for Annotation-Free Open-Set Semantic Segmentation of Aerial Point Clouds with Domain Shifts
Semantic segmentation of aerial point cloud is trapped in a generalization crisis under distinct domain shifts. While test-time adaptation offers a privacy-preserving and computationally efficient way to adapt pre-traine…
Point Cloud SegmentationSemantic SegmentationTest-time AdaptationPoint Clouds