3D-Consistent Image Inpainting with Diffusion Models
We address the problem of 3D inconsistency of image inpainting based on diffusion models. We propose a generative model using image pairs that belong to the same scene. To achieve the 3D-consistent and semantically coherent inpainting, we modify the generative diffusion model by incorporating an alternative point of view of the scene into the denoising process. This creates an inductive bias that allows to recover 3D priors while training to denoise in 2D, without explicit 3D supervision. Training unconditional diffusion models with additional images as in-context guidance allows to harmonize the masked and non-masked regions while repainting and ensures the 3D consistency. We evaluate our method on one synthetic and three real-world datasets and show that it generates semantically coherent and 3D-consistent inpaintings and outperforms the state-of-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
DenoisingImage InpaintingInductive BiasMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MVIP-NeRF: Multi-view 3D Inpainting on NeRF Scenes via Diffusion Prior
Despite the emergence of successful NeRF inpainting methods built upon explicit RGB and depth 2D inpainting supervisions, these methods are inherently constrained by the capabilities of their underlying 2D inpainters. Th…
3D InpaintingNeRFHierarchical Diffusion Framework for Pseudo-Healthy Brain MRI Inpainting with Enhanced 3D Consistency
Pseudo-healthy image inpainting is an essential preprocessing step for analyzing pathological brain MRI scans. Most current inpainting methods favor slice-wise 2D models for their high in-plane fidelity, but their indepe…
Image InpaintingTurboFill: Adapting Few-step Text-to-image Model for Fast Image Inpainting
This paper introduces TurboFill, a fast image inpainting model that enhances a few-step text-to-image diffusion model with an inpainting adapter for high-quality and efficient inpainting. While standard diffusion mod…
Image InpaintingGuidPaint: Class-Guided Image Inpainting with Diffusion Models
In recent years, diffusion models have been widely adopted for image inpainting tasks due to their powerful generative capabilities, achieving impressive results. Existing multimodal inpainting methods based on diffusion…
Image InpaintingAmortized Guidance for Image Inpainting with Pretrained Diffusion Models
We study image inpainting with generative diffusion models. Existing methods typically either train dedicated task-specific models, or adapt a pretrained diffusion model separately for each masked image at deployment. We…
Image Inpainting