Nerflets: Local Radiance Fields for Efficient Structure-Aware 3D Scene Representation from 2D Supervision
We address efficient and structure-aware 3D scene representation from images. Nerflets are our key contribution -- a set of local neural radiance fields that together represent a scene. Each nerflet maintains its own spatial position, orientation, and extent, within which it contributes to panoptic, density, and radiance reconstructions. By leveraging only photometric and inferred panoptic image supervision, we can directly and jointly optimize the parameters of a set of nerflets so as to form a decomposed representation of the scene, where each object instance is represented by a group of nerflets. During experiments with indoor and outdoor environments, we find that nerflets: (1) fit and approximate the scene more efficiently than traditional global NeRFs, (2) allow the extraction of panoptic and photometric renderings from arbitrary views, and (3) enable tasks rare for NeRFs, such as 3D panoptic segmentation and interactive editing.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Panoptic SegmentationPanoptic SegmentationPositionSimilar Papers 제목 키워드 기반
DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields
The accurate reconstruction of dynamic scenes with neural radiance fields is significantly dependent on the estimation of camera poses. Widely used structure-from-motion pipelines encounter difficulties in accurately tra…
Camera LocalizationCamera Pose EstimationDynamic ReconstructionNeRF+4360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360$^\circ$ Radiance Fields
Virtual tour among sparse 360$^\circ$ images is widely used while hindering smooth and immersive roaming experiences. The emergence of Neural Radiance Field (NeRF) has showcased significant progress in synthesizing novel…
NeRFNovel View SynthesisGenerative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects
3D-aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images even for topology-varying object categories. However, these meth…
DisentanglementImage GenerationNeRFObjectSNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning
As previous representations for reinforcement learning cannot effectively incorporate a human-intuitive understanding of the 3D environment, they usually suffer from sub-optimal performances. In this paper, we present Se…
3D ReconstructionNeRFNovel View Synthesisreinforcement-learning+2Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields
Inverse rendering aims at recovering both geometry and materials of objects. It provides a more compatible reconstruction for conventional rendering engines, compared with the neural radiance fields (NeRFs). On the other…
Inverse RenderingNeRFObject