DatasetNeRF: Efficient 3D-aware Data Factory with Generative Radiance Fields
Progress in 3D computer vision tasks demands a huge amount of data, yet annotating multi-view images with 3D-consistent annotations, or point clouds with part segmentation is both time-consuming and challenging. This paper introduces DatasetNeRF, a novel approach capable of generating infinite, high-quality 3D-consistent 2D annotations alongside 3D point cloud segmentations, while utilizing minimal 2D human-labeled annotations. Specifically, we leverage the strong semantic prior within a 3D generative model to train a semantic decoder, requiring only a handful of fine-grained labeled samples. Once trained, the decoder efficiently generalizes across the latent space, enabling the generation of infinite data. The generated data is applicable across various computer vision tasks, including video segmentation and 3D point cloud segmentation. Our approach not only surpasses baseline models in segmentation quality, achieving superior 3D consistency and segmentation precision on individual images, but also demonstrates versatility by being applicable to both articulated and non-articulated generative models. Furthermore, we explore applications stemming from our approach, such as 3D-aware semantic editing and 3D inversion.
Code (1)
Tasks
DecoderPoint Cloud SegmentationSegmentationVideo SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects
3D-aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images even for topology-varying object categories. However, these meth…
DisentanglementImage GenerationNeRFObjectGenerative Occupancy Fields for 3D Surface-Aware Image Synthesis
The advent of generative radiance fields has significantly promoted the development of 3D-aware image synthesis. The cumulative rendering process in radiance fields makes training these generative models much easier sinc…
3D-Aware Image SynthesisImage GenerationObjectGRAM: Generative Radiance Manifolds for 3D-Aware Image Generation
3D-aware image generative modeling aims to generate 3D-consistent images with explicitly controllable camera poses. Recent works have shown promising results by training neural radiance field (NeRF) generators on unstruc…
3D-Aware Image SynthesisFace GenerationImage GenerationNeRF+1SemFaceEdit: Semantic Face Editing on Generative Radiance Manifolds
Despite multiple view consistency offered by 3D-aware GAN techniques, the resulting images often lack the capacity for localized editing. In response, generative radiance manifolds emerge as an efficient approach for con…
DisentanglementGenerative Neural Articulated Radiance Fields
Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress. These 3D GANs, however, have not been demonstrated for h…