CUS3D :CLIP-based Unsupervised 3D Segmentation via Object-level Denoise
To ease the difficulty of acquiring annotation labels in 3D data, a common method is using unsupervised and open-vocabulary semantic segmentation, which leverage 2D CLIP semantic knowledge. In this paper, unlike previous research that ignores the `noise'' raised during feature projection from 2D to 3D, we propose a novel distillation learning framework named CUS3D. In our approach, an object-level denosing projection module is designed to screen out the `noise'' and ensure more accurate 3D feature. Based on the obtained features, a multimodal distillation learning module is designed to align the 3D feature with CLIP semantic feature space with object-centered constrains to achieve advanced unsupervised semantic segmentation. We conduct comprehensive experiments in both unsupervised and open-vocabulary segmentation, and the results consistently showcase the superiority of our model in achieving advanced unsupervised segmentation results and its effectiveness in open-vocabulary segmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Open Vocabulary Semantic SegmentationOpen-Vocabulary Semantic SegmentationSegmentationSemantic SegmentationUnsupervised Semantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CLIP-DIY: CLIP Dense Inference Yields Open-Vocabulary Semantic Segmentation For-Free
The emergence of CLIP has opened the way for open-world image perception. The zero-shot classification capabilities of the model are impressive but are harder to use for dense tasks such as image segmentation. Several me…
Image SegmentationObject LocalizationOpen Vocabulary Semantic SegmentationOpen-Vocabulary Semantic Segmentation+6Learn to Rectify the Bias of CLIP for Unsupervised Semantic Segmentation
Recent works utilize CLIP to perform the challenging unsupervised semantic segmentation task where only images without annotations are available. However we observe that when adopting CLIP to such a pixel-level under…
SegmentationSemantic SegmentationUnsupervised Semantic SegmentationSegmentation-Aware Image Denoising without Knowing True Segmentation
Several recent works discussed application-driven image restoration neural networks, which are capable of not only removing noise in images but also preserving their semantic-aware details, making them suitable for vario…
DenoisingImage DenoisingImage RestorationSegmentation+1A Training-Free Framework for Open-Vocabulary Image Segmentation and Recognition with EfficientNet and CLIP
This paper presents a novel training-free framework for open-vocabulary image segmentation and object recognition (OVSR), which leverages EfficientNetB0, a convolutional neural network, for unsupervised segmentation and …
Object RecognitionImage SegmentationUVIS: Unsupervised Video Instance Segmentation
Video instance segmentation requires classifying, segmenting, and tracking every object across video frames. Unlike existing approaches that rely on masks, boxes, or category labels, we propose UVIS, a novel Unsupervised…
Instance SegmentationLanguage ModellingPseudo LabelSegmentation+2