CL-NeRF: Continual Learning of Neural Radiance Fields for Evolving Scene Representation

2023-09-21 · NeurIPS 2023 11

Existing methods for adapting Neural Radiance Fields (NeRFs) to scene changes require extensive data capture and model retraining, which is both time-consuming and labor-intensive. In this paper, we tackle the challenge of efficiently adapting NeRFs to real-world scene changes over time using a few new images while retaining the memory of unaltered areas, focusing on the continual learning aspect of NeRFs. To this end, we propose CL-NeRF, which consists of two key components: a lightweight expert adaptor for adapting to new changes and evolving scene representations and a conflict-aware knowledge distillation learning objective for memorizing unchanged parts. We also present a new benchmark for evaluating Continual Learning of NeRFs with comprehensive metrics. Our extensive experiments demonstrate that CL-NeRF can synthesize high-quality novel views of both changed and unchanged regions with high training efficiency, surpassing existing methods in terms of reducing forgetting and adapting to changes. Code and benchmark will be made available.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

Instant Continual Learning of Neural Radiance Fields

2023-09-04 · Ryan Po, Zhengyang Dong, Alexander W. Bergman, Gordon Wetzstein

Neural radiance fields (NeRFs) have emerged as an effective method for novel-view synthesis and 3D scene reconstruction. However, conventional training methods require access to all training views during scene optimizati…

3D Scene Reconstruction Continual Learning NeRF Novel View Synthesis

SCARF: Scalable Continual Learning Framework for Memory-efficient Multiple Neural Radiance Fields

2024-09-06 · Yuze Wang, Junyi Wang, Chen Wang, Wantong Duan 외

This paper introduces a novel continual learning framework for synthesising novel views of multiple scenes, learning multiple 3D scenes incrementally, and updating the network parameters only with the training data of th…

Continual Learning Knowledge Distillation NeRF

Incremental Multi-Scene Modeling via Continual Neural Graphics Primitives

2024-11-29 · Prajwal Singh, Ashish Tiwari, Gautam Vashishtha, Shanmuganathan Raman

Neural radiance fields (NeRF) have revolutionized photorealistic rendering of novel views for 3D scenes. Despite their growing popularity and efficiency as 3D resources, NeRFs face scalability challenges due to the need …

Continual Learning NeRF

Self-Evolving Neural Radiance Fields

2023-12-02 · Jaewoo Jung, Jisang Han, Jiwon Kang, Seongchan Kim 외

Recently, neural radiance field (NeRF) has shown remarkable performance in novel view synthesis and 3D reconstruction. However, it still requires abundant high-quality images, limiting its applicability in real-world sce…

3D Reconstruction NeRF Novel View Synthesis

Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations

2023-10-27 · Tristan Aumentado-Armstrong, Ashkan Mirzaei, Marcus A. Brubaker, Jonathan Kelly 외

Neural Radiance Fields (NeRFs) have proven to be powerful 3D representations, capable of high quality novel view synthesis of complex scenes. While NeRFs have been applied to graphics, vision, and robotics, problems with…

Continual Learning NeRF Novel View Synthesis