3D Reconstruction with Generalizable Neural Fields using Scene Priors

2023-09-26 · Yang Fu, Shalini De Mello, Xueting Li, Amey Kulkarni, Jan Kautz, Xiaolong Wang, Sifei Liu

High-fidelity 3D scene reconstruction has been substantially advanced by recent progress in neural fields. However, most existing methods train a separate network from scratch for each individual scene. This is not scalable, inefficient, and unable to yield good results given limited views. While learning-based multi-view stereo methods alleviate this issue to some extent, their multi-view setting makes it less flexible to scale up and to broad applications. Instead, we introduce training generalizable Neural Fields incorporating scene Priors (NFPs). The NFP network maps any single-view RGB-D image into signed distance and radiance values. A complete scene can be reconstructed by merging individual frames in the volumetric space WITHOUT a fusion module, which provides better flexibility. The scene priors can be trained on large-scale datasets, allowing for fast adaptation to the reconstruction of a new scene with fewer views. NFP not only demonstrates SOTA scene reconstruction performance and efficiency, but it also supports single-image novel-view synthesis, which is underexplored in neural fields. More qualitative results are available at: https://oasisyang.github.io/neural-prior

📄 PDF Abstract BibTeX arXiv:2309.15164

Code (0)

등록된 구현이 없습니다.

Tasks

3D Reconstruction3D Scene ReconstructionNovel View Synthesis

HyP-NeRF: Learning Improved NeRF Priors using a HyperNetwork

2023-06-09 · NeurIPS 2023 11 · Bipasha Sen, Gaurav Singh, Aditya Agarwal, Rohith Agaram 외

Neural Radiance Fields (NeRF) have become an increasingly popular representation to capture high-quality appearance and shape of scenes and objects. However, learning generalizable NeRF priors over categories of scenes o…

NeRF Retrieval

GeoTransfer : Generalizable Few-Shot Multi-View Reconstruction via Transfer Learning

2024-08-27 · Shubhendu Jena, Franck Multon, Adnane Boukhayma

This paper presents a novel approach for sparse 3D reconstruction by leveraging the expressive power of Neural Radiance Fields (NeRFs) and fast transfer of their features to learn accurate occupancy fields. Existing 3D r…

3D Reconstruction NeRF Surface Reconstruction Transfer Learning

G3R: Gradient Guided Generalizable Reconstruction

2024-09-28 · Yun Chen, Jingkang Wang, Ze Yang, Sivabalan Manivasagam 외

Large scale 3D scene reconstruction is important for applications such as virtual reality and simulation. Existing neural rendering approaches (e.g., NeRF, 3DGS) have achieved realistic reconstructions on large scenes, b…

3DGS 3D Scene Reconstruction NeRF Neural Rendering

Entangled View-Epipolar Information Aggregation for Generalizable Neural Radiance Fields

2023-11-20 · CVPR 2024 1 · Zhiyuan Min, Yawei Luo, Wei Yang, Yuesong Wang 외

Generalizable NeRF can directly synthesize novel views across new scenes, eliminating the need for scene-specific retraining in vanilla NeRF. A critical enabling factor in these approaches is the extraction of a generali…

Generalizable Novel View Synthesis NeRF

MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo

2021-03-29 · ICCV 2021 10 · Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang 외

We present MVSNeRF, a novel neural rendering approach that can efficiently reconstruct neural radiance fields for view synthesis. Unlike prior works on neural radiance fields that consider per-scene optimization on dense…

NeRF Neural Rendering