Interactive Segmentation of Radiance Fields
Radiance Fields (RF) are popular to represent casually-captured scenes for new view synthesis and several applications beyond it. Mixed reality on personal spaces needs understanding and manipulating scenes represented as RFs, with semantic segmentation of objects as an important step. Prior segmentation efforts show promise but don't scale to complex objects with diverse appearance. We present the ISRF method to interactively segment objects with fine structure and appearance. Nearest neighbor feature matching using distilled semantic features identifies high-confidence seed regions. Bilateral search in a joint spatio-semantic space grows the region to recover accurate segmentation. We show state-of-the-art results of segmenting objects from RFs and compositing them to another scene, changing appearance, etc., and an interactive segmentation tool that others can use. Project Page: https://rahul-goel.github.io/isrf/
Code (0)
등록된 구현이 없습니다.
Tasks
Interactive SegmentationMixed RealitySegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Scene-Generalizable Interactive Segmentation of Radiance Fields
Existing methods for interactive segmentation in radiance fields entail scene-specific optimization and thus cannot generalize across different scenes, which greatly limits their applicability. In this work we make the f…
Interactive SegmentationSegmentationSemantic SegmentationEfficient Neural Radiance Fields for Interactive Free-viewpoint Video
This paper aims to tackle the challenge of efficiently producing interactive free-viewpoint videos. Some recent works equip neural radiance fields with image encoders, enabling them to generalize across scenes. When proc…
Depth EstimationDepth PredictionNovel View SynthesisInteractive Geometry Editing of Neural Radiance Fields
In this paper, we propose a method that enables interactive geometry editing for neural radiance fields manipulation. We use two proxy cages(inner cage and outer cage) to edit a scene. The inner cage defines the operatio…
3D geometryNerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervision
We address efficient and structure-aware 3D scene representation from images. Nerflets are our key contribution -- a set of local neural radiance fields that together represent a scene. Each nerflet maintains its own spa…
3D Panoptic SegmentationPanoptic SegmentationPositionLiveScene: Language Embedding Interactive Radiance Fields for Physical Scene Rendering and Control
This paper scales object-level reconstruction to complex scenes, advancing interactive scene reconstruction. We introduce two datasets, OmniSim and InterReal, featuring 28 scenes with multiple interactive objects. To tac…
Novel View SynthesisObjectObject Reconstruction