3DFill:Reference-guided Image Inpainting by Self-supervised 3D Image Alignment
Most existing image inpainting algorithms are based on a single view, struggling with large holes or the holes containing complicated scenes. Some reference-guided algorithms fill the hole by referring to another viewpoint image and use 2D image alignment. Due to the camera imaging process, simple 2D transformation is difficult to achieve a satisfactory result. In this paper, we propose 3DFill, a simple and efficient method for reference-guided image inpainting. Given a target image with arbitrary hole regions and a reference image from another viewpoint, the 3DFill first aligns the two images by a two-stage method: 3D projection + 2D transformation, which has better results than 2D image alignment. The 3D projection is an overall alignment between images and the 2D transformation is a local alignment focused on the hole region. The entire process of image alignment is self-supervised. We then fill the hole in the target image with the contents of the aligned image. Finally, we use a conditional generation network to refine the filled image to obtain the inpainting result. 3DFill achieves state-of-the-art performance on image inpainting across a variety of wide view shifts and has a faster inference speed than other inpainting models.
Code (0)
등록된 구현이 없습니다.
Tasks
Image InpaintingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Generator Pyramid for High-Resolution Image Inpainting
Inpainting high-resolution images with large holes challenges existing deep learning based image inpainting methods. We present a novel framework -- PyramidFill for high-resolution image inpainting task, which explicitly…
Image InpaintingTexture SynthesisVocal Bursts Intensity PredictionDfilled: Repurposing Edge-Enhancing Diffusion for Guided DSM Void Filling
Digital Surface Models (DSMs) are essential for accurately representing Earth's topography in geospatial analyses. DSMs capture detailed elevations of natural and manmade features, crucial for applications like urban pla…
3D ReconstructionSuper-ResolutionCoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying
Image inpainting aims to fill the missing hole of the input. It is hard to solve this task efficiently when facing high-resolution images due to two reasons: (1) Large reception field needs to be handled for high-resolut…
GPUImage InpaintingVocal Bursts Intensity PredictionReference-Guided Texture and Structure Inference for Image Inpainting
Existing learning-based image inpainting methods are still in challenge when facing complex semantic environments and diverse hole patterns. The prior information learned from the large scale training data is still insuf…
DecoderImage InpaintingReference-Guided Large-Scale Face Inpainting with Identity and Texture Control
Face inpainting aims at plausibly predicting missing pixels of face images within a corrupted region. Most existing methods rely on generative models learning a face image distribution from a big dataset, which produces …
Facial Inpainting