paper-with-me

홈 › Papers

LaRa: Efficient Large-Baseline Radiance Fields

2024-07-05 · Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, Andreas Geiger

Radiance field methods have achieved photorealistic novel view synthesis and geometry reconstruction. But they are mostly applied in per-scene optimization or small-baseline settings. While several recent works investigate feed-forward reconstruction with large baselines by utilizing transformers, they all operate with a standard global attention mechanism and hence ignore the local nature of 3D reconstruction. We propose a method that unifies local and global reasoning in transformer layers, resulting in improved quality and faster convergence. Our model represents scenes as Gaussian Volumes and combines this with an image encoder and Group Attention Layers for efficient feed-forward reconstruction. Experimental results demonstrate that our model, trained for two days on four GPUs, demonstrates high fidelity in reconstructing 360 deg radiance fields, and robustness to zero-shot and out-of-domain testing. Our project Page: https://apchenstu.github.io/LaRa/.

📄 PDF Abstract BibTeX arXiv:2407.04699

Code (1)

autonomousvision/lara jax

Tasks

3D ReconstructionNovel View Synthesis

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음

Similar Papers 제목 키워드 기반

SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Images

2022-12-18 · Abdullah Hamdi, Bernard Ghanem, Matthias Nießner

Recent advances in Neural Radiance Fields (NeRFs) treat the problem of novel view synthesis as Sparse Radiance Field (SRF) optimization using sparse voxels for efficient and fast rendering (plenoxels,InstantNGP). In orde…

Novel View Synthesis

PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas

2023-06-02 · NeurIPS 2023 11 · Zheng Chen, Yan-Pei Cao, Yuan-Chen Guo, Chen Wang 외

Achieving an immersive experience enabling users to explore virtual environments with six degrees of freedom (6DoF) is essential for various applications such as virtual reality (VR). Wide-baseline panoramas are commonly…

Depth Estimation

Deformable Neural Radiance Fields using RGB and Event Cameras

2023-09-15 · ICCV 2023 1 · Qi Ma, Danda Pani Paudel, Ajad Chhatkuli, Luc van Gool

Modeling Neural Radiance Fields for fast-moving deformable objects from visual data alone is a challenging problem. A major issue arises due to the high deformation and low acquisition rates. To address this problem, we …

Self-Aligning Depth-regularized Radiance Fields for Asynchronous RGB-D Sequences

2022-11-14 · Yuxin Huang, Andong Yang, Zirui Wu, Yuantao Chen 외

It has been shown that learning radiance fields with depth rendering and depth supervision can effectively promote the quality and convergence of view synthesis. However, this paradigm requires input RGB-D sequences to b…

Autonomous DrivingBenchmarking

360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360$^\circ$ Radiance Fields

2022-08-04 · Huajian Huang, Yingshu Chen, Tianjia Zhang, Sai-Kit Yeung

Virtual tour among sparse 360$^\circ$ images is widely used while hindering smooth and immersive roaming experiences. The emergence of Neural Radiance Field (NeRF) has showcased significant progress in synthesizing novel…

NeRFNovel View Synthesis