DINER: Depth-aware Image-based NEural Radiance fields
We present Depth-aware Image-based NEural Radiance fields (DINER). Given a sparse set of RGB input views, we predict depth and feature maps to guide the reconstruction of a volumetric scene representation that allows us to render 3D objects under novel views. Specifically, we propose novel techniques to incorporate depth information into feature fusion and efficient scene sampling. In comparison to the previous state of the art, DINER achieves higher synthesis quality and can process input views with greater disparity. This allows us to capture scenes more completely without changing capturing hardware requirements and ultimately enables larger viewpoint changes during novel view synthesis. We evaluate our method by synthesizing novel views, both for human heads and for general objects, and observe significantly improved qualitative results and increased perceptual metrics compared to the previous state of the art. The code is publicly available for research purposes.
Code (0)
등록된 구현이 없습니다.
Tasks
Novel View SynthesisSimilar Papers 제목 키워드 기반
Simple-RF: Regularizing Sparse Input Radiance Fields with Simpler Solutions
Neural Radiance Fields (NeRF) show impressive performance in photo-realistic free-view rendering of scenes. Recent improvements on the NeRF such as TensoRF and ZipNeRF employ explicit models for faster optimization and r…
NeRFDisorder-invariant Implicit Neural Representation
Implicit neural representation (INR) characterizes the attributes of a signal as a function of corresponding coordinates which emerges as a sharp weapon for solving inverse problems. However, the expressive power of INR …
AttributeRetrievalSelf-Aligning Depth-regularized Radiance Fields for Asynchronous RGB-D Sequences
It has been shown that learning radiance fields with depth rendering and depth supervision can effectively promote the quality and convergence of view synthesis. However, this paradigm requires input RGB-D sequences to b…
Autonomous DrivingBenchmarkingSNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning
As previous representations for reinforcement learning cannot effectively incorporate a human-intuitive understanding of the 3D environment, they usually suffer from sub-optimal performances. In this paper, we present Se…
3D ReconstructionNeRFNovel View Synthesisreinforcement-learning+2SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields
3D reconstruction from a single 2D image was extensively covered in the literature but relies on depth supervision at training time, which limits its applicability. To relax the dependence to depth we propose SceneRF, a …
3D Reconstruction3D Scene ReconstructionAutonomous DrivingDepth Estimation+4