paper-with-me

홈 › Papers

DATENeRF: Depth-Aware Text-based Editing of NeRFs

2024-04-06 · Sara Rojas, Julien Philip, Kai Zhang, Sai Bi, Fujun Luan, Bernard Ghanem, Kalyan Sunkavall

Recent advancements in diffusion models have shown remarkable proficiency in editing 2D images based on text prompts. However, extending these techniques to edit scenes in Neural Radiance Fields (NeRF) is complex, as editing individual 2D frames can result in inconsistencies across multiple views. Our crucial insight is that a NeRF scene's geometry can serve as a bridge to integrate these 2D edits. Utilizing this geometry, we employ a depth-conditioned ControlNet to enhance the coherence of each 2D image modification. Moreover, we introduce an inpainting approach that leverages the depth information of NeRF scenes to distribute 2D edits across different images, ensuring robustness against errors and resampling challenges. Our results reveal that this methodology achieves more consistent, lifelike, and detailed edits than existing leading methods for text-driven NeRF scene editing.

📄 PDF Abstract BibTeX arXiv:2404.04526

Code (0)

등록된 구현이 없습니다.

Tasks

NeRF

Methods 이 논문이 사용한 방법론

Inpainting Train a convolutional neural network to generate the contents of an arbitrary image region conditioned on its surroundings.
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context

2023-10-15 · Binglun Wang, Niladri Shekhar Dutt, Niloy J. Mitra

Neural Radiance Fields (NeRFs) have recently emerged as a popular option for photo-realistic object capture due to their ability to faithfully capture high-fidelity volumetric content even from handheld video input. Alth…

NeRF

ICE-NeRF: Interactive Color Editing of NeRFs via Decomposition-Aware Weight Optimization

2023-01-01 · ICCV 2023 1 · Jae-Hyeok Lee, Dae-shik Kim

Neural Radiance Fields (NeRFs) have gained considerable attention for their high-quality results in 3D scene reconstruction and rendering. Recently, there have been active studies on various tasks such as novel view …

3D Scene ReconstructionNeRFNovel View Synthesis

GenN2N: Generative NeRF2NeRF Translation

2024-04-03 · CVPR 2024 1 · Xiangyue Liu, Han Xue, Kunming Luo, Ping Tan 외

We present GenN2N, a unified NeRF-to-NeRF translation framework for various NeRF translation tasks such as text-driven NeRF editing, colorization, super-resolution, inpainting, etc. Unlike previous methods designed for i…

ColorizationContrastive LearningNeRFSuper-Resolution+1

Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Review

2024-02-17 · Thang-Anh-Quan Nguyen, Amine Bourki, Mátyás Macudzinski, Anthony Brunel 외

This review thoroughly examines the role of semantically-aware Neural Radiance Fields (NeRFs) in visual scene understanding, covering an analysis of over 250 scholarly papers. It explores how NeRFs adeptly infer 3D repre…

Panoptic SegmentationScene SegmentationScene UnderstandingSegmentation

FaceDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models

2023-06-01 · NeurIPS 2023 11 · Hao Zhang, Yanbo Xu, Tianyuan Dai, Yu-Wing Tai 외

The ability to create high-quality 3D faces from a single image has become increasingly important with wide applications in video conferencing, AR/VR, and advanced video editing in movie industries. In this paper, we pro…

3D Face ReconstructionFace ReconstructionNeRFVideo Editing+1