홈 › Papers

iNVS: Repurposing Diffusion Inpainters for Novel View Synthesis

2023-10-24 · Yash Kant, Aliaksandr Siarohin, Michael Vasilkovsky, Riza Alp Guler, Jian Ren, Sergey Tulyakov, Igor Gilitschenski

We present a method for generating consistent novel views from a single source image. Our approach focuses on maximizing the reuse of visible pixels from the source image. To achieve this, we use a monocular depth estimator that transfers visible pixels from the source view to the target view. Starting from a pre-trained 2D inpainting diffusion model, we train our method on the large-scale Objaverse dataset to learn 3D object priors. While training we use a novel masking mechanism based on epipolar lines to further improve the quality of our approach. This allows our framework to perform zero-shot novel view synthesis on a variety of objects. We evaluate the zero-shot abilities of our framework on three challenging datasets: Google Scanned Objects, Ray Traced Multiview, and Common Objects in 3D. See our webpage for more details: https://yashkant.github.io/invs/

📄 PDF Abstract BibTeX arXiv:2310.16167

Code (0)

등록된 구현이 없습니다.

Tasks

Novel View Synthesis

Methods 이 논문이 사용한 방법론

Inpainting Train a convolutional neural network to generate the contents of an arbitrary image region conditioned on its surroundings.

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

InvSplat: Inverse Feed-Forward Scene Splatting

2026-07-02 · Polina Karpikova, Wenjing Bian, Haofei Xu, Hendrik Lensch 외 arxiv

Inverse rendering aims to recover both 3D geometry and physically meaningful material properties from images, enabling applications such as relighting and novel view synthesis. Optimization-based methods achieve high fid…

Multi-View 3D Reconstruction Novel View Synthesis Inverse Rendering

Benchmarking Single-Step Inpainting Methods for Multi-Object 3D Gaussian Splatting Scenes

2026-05-29 · Finn Dröge, Cecilia Curreli, Abhishek Saroha, Daniel Cremers arxiv

The tasks of object removal and inpainting 3D Gaussian Splatting (3DGS) scenes face challenges such as 3D consistency across camera views. In comparing 2D inpainters and their suitability for the 3D domain, we find that …

Disrupting Diffusion-based Inpainters with Semantic Digression

2024-07-14 · Geonho Son, Juhun Lee, Simon S. Woo

The fabrication of visual misinformation on the web and social media has increased exponentially with the advent of foundational text-to-image diffusion models. Namely, Stable Diffusion inpainters allow the synthesis of …

GPU Misinformation

InvSeg: Test-Time Prompt Inversion for Semantic Segmentation

2024-10-15 · Jiayi Lin, Jiabo Huang, Jian Hu, Shaogang Gong

Visual-textual correlations in the attention maps derived from text-to-image diffusion models are proven beneficial to dense visual prediction tasks, e.g., semantic segmentation. However, a significant challenge arises d…

Image Generation Open Vocabulary Semantic Segmentation Open-Vocabulary Semantic Segmentation Segmentation+1

MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Prior

2024-05-05 · CVPR 2024 1 · Honghua Chen, Chen Change Loy, Xingang Pan

Despite the emergence of successful NeRF inpainting methods built upon explicit RGB and depth 2D inpainting supervisions, these methods are inherently constrained by the capabilities of their underlying 2D inpainters. Th…

3D Inpainting NeRF