OpenSplat3D: Open-Vocabulary 3D Instance Segmentation using Gaussian Splatting
3D Gaussian Splatting (3DGS) has emerged as a powerful representation for neural scene reconstruction, offering high-quality novel view synthesis while maintaining computational efficiency. In this paper, we extend the capabilities of 3DGS beyond pure scene representation by introducing an approach for open-vocabulary 3D instance segmentation without requiring manual labeling, termed OpenSplat3D. Our method leverages feature-splatting techniques to associate semantic information with individual Gaussians, enabling fine-grained scene understanding. We incorporate Segment Anything Model instance masks with a contrastive loss formulation as guidance for the instance features to achieve accurate instance-level segmentation. Furthermore, we utilize language embeddings of a vision-language model, allowing for flexible, text-driven instance identification. This combination enables our system to identify and segment arbitrary objects in 3D scenes based on natural language descriptions. We show results on LERF-mask and LERF-OVS as well as the full ScanNet++ validation set, demonstrating the effectiveness of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
3DGS3D Instance SegmentationComputational EfficiencyFeature SplattingInstance SegmentationLanguage ModelingLanguage ModellingNovel View Synthesisopen vocabulary 3d instance segmentationScene UnderstandingSemantic SegmentationSimilar Papers 제목 키워드 기반
Relation-Centric Open-Vocabulary 3D Gaussian Segmentation
Open-vocabulary 3D Gaussian segmentation is challenging because it requires language understanding for diverse queries and accurate separation of Gaussians along object boundaries. Prior approaches either embed language …
GLS: Geometry-aware 3D Language Gaussian Splatting
Recently, 3D Gaussian Splatting (3DGS) has achieved significant performance on indoor surface reconstruction and open-vocabulary segmentation. This paper presents GLS, a unified framework of surface reconstruction and op…
3DGSSegmentationSurface ReconstructionPanoGS: Gaussian-based Panoptic Segmentation for 3D Open Vocabulary Scene Understanding
Recently, 3D Gaussian Splatting (3DGS) has shown encouraging performance for open vocabulary scene understanding tasks. However, previous methods cannot distinguish 3D instance-level information, which usually predicts a…
3DGSDecoderGraph ClusteringPanoptic Segmentation+1OVGaussian: Generalizable 3D Gaussian Segmentation with Open Vocabularies
Open-vocabulary scene understanding using 3D Gaussian (3DGS) representations has garnered considerable attention. However, existing methods mostly lift knowledge from large 2D vision models into 3DGS on a scene-by-scene …
3DGS3D Semantic SegmentationOpen Vocabulary Semantic SegmentationOpen-Vocabulary Semantic Segmentation+2OpenInsGaussian: Open-vocabulary Instance Gaussian Segmentation with Context-aware Cross-view Fusion
Understanding 3D scenes is pivotal for autonomous driving, robotics, and augmented reality. Recent semantic Gaussian Splatting approaches leverage large-scale 2D vision models to project 2D semantic features onto 3D scen…
Scene UnderstandingAutonomous Driving