Entangled View-Epipolar Information Aggregation for Generalizable Neural Radiance Fields
Generalizable NeRF can directly synthesize novel views across new scenes, eliminating the need for scene-specific retraining in vanilla NeRF. A critical enabling factor in these approaches is the extraction of a generalizable 3D representation by aggregating source-view features. In this paper, we propose an Entangled View-Epipolar Information Aggregation method dubbed EVE-NeRF. Different from existing methods that consider cross-view and along-epipolar information independently, EVE-NeRF conducts the view-epipolar feature aggregation in an entangled manner by injecting the scene-invariant appearance continuity and geometry consistency priors to the aggregation process. Our approach effectively mitigates the potential lack of inherent geometric and appearance constraint resulting from one-dimensional interactions, thus further boosting the 3D representation generalizablity. EVE-NeRF attains state-of-the-art performance across various evaluation scenarios. Extensive experiments demonstate that, compared to prevailing single-dimensional aggregation, the entangled network excels in the accuracy of 3D scene geometry and appearance reconstruction. Our code is publicly available at https://github.com/tatakai1/EVENeRF.
Code (1)
Tasks
Generalizable Novel View SynthesisNeRFSimilar Papers 제목 키워드 기반
Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis
Generalizable 3D Gaussian splitting (3DGS) can reconstruct new scenes from sparse-view observations in a feed-forward inference manner, eliminating the need for scene-specific retraining required in conventional 3DGS. Ho…
3DGSGeneralizable Novel View SynthesisNovel View SynthesisMulti-View Industrial Anomaly Detection with Epipolar Constrained Cross-View Fusion
Multi-camera systems provide richer contextual information for industrial anomaly detection. However, traditional methods process each view independently, disregarding the complementary information across viewpoints. Exi…
Anomaly DetectionNeural Surface Reconstruction from Sparse Views Using Epipolar Geometry
This paper addresses the challenge of reconstructing surfaces from sparse view inputs, where ambiguity and occlusions due to missing information pose significant hurdles. We present a novel approach, named EpiS, that inc…
Depth EstimationMonocular Depth EstimationSurface ReconstructionTripletWT-MVSNet: Window-based Transformers for Multi-view Stereo
Recently, Transformers were shown to enhance the performance of multi-view stereo by enabling long-range feature interaction. In this work, we propose Window-based Transformers (WT) for local feature matching and global …
When Epipolar Constraint Meets Non-local Operators in Multi-View Stereo
Learning-based multi-view stereo (MVS) method heavily relies on feature matching, which requires distinctive and descriptive representations. An effective solution is to apply non-local feature aggregation, e.g., Transfo…
3D ReconstructionDescriptivePoint CloudsStereo Matching