NÜWA-LIP: Language Guided Image Inpainting with Defect-free VQGAN
Language guided image inpainting aims to fill in the defective regions of an image under the guidance of text while keeping non-defective regions unchanged. However, the encoding process of existing models suffers from either receptive spreading of defective regions or information loss of non-defective regions, giving rise to visually unappealing inpainting results. To address the above issues, this paper proposes N\"UWA-LIP by incorporating defect-free VQGAN (DF-VQGAN) with multi-perspective sequence to sequence (MP-S2S). In particular, DF-VQGAN introduces relative estimation to control receptive spreading and adopts symmetrical connections to protect information. MP-S2S further enhances visual information from complementary perspectives, including both low-level pixels and high-level tokens. Experiments show that DF-VQGAN performs more robustness than VQGAN. To evaluate the inpainting performance of our model, we built up 3 open-domain benchmarks, where N\"UWA-LIP is also superior to recent strong baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Image InpaintingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
NUWA-LIP: Language-Guided Image Inpainting With Defect-Free VQGAN
Language-guided image inpainting aims to fill the defective regions of an image under the guidance of text while keeping the non-defective regions unchanged. However, directly encoding the defective images is prone t…
Image InpaintingDefectFill: Realistic Defect Generation with Inpainting Diffusion Model for Visual Inspection
Developing effective visual inspection models remains challenging due to the scarcity of defect data. While image generation models have been used to synthesize defect images, producing highly realistic defects remains d…
Image GenerationFreeInpaint: Tuning-free Prompt Alignment and Visual Rationality Enhancement in Image Inpainting
Text-guided image inpainting endeavors to generate new content within specified regions of images using textual prompts from users. The primary challenge is to accurately align the inpainted areas with the user-provided …
Image InpaintingToken Painter: Training-Free Text-Guided Image Inpainting via Mask Autoregressive Models
Text-guided image inpainting aims to inpaint masked image regions based on a textual prompt while preserving the background. Although diffusion-based methods have become dominant, their property of modeling the entire im…
Image InpaintingDiffree: Text-Guided Shape Free Object Inpainting with Diffusion Model
This paper addresses an important problem of object addition for images with only text guidance. It is challenging because the new object must be integrated seamlessly into the image with consistent visual context, such …
Image InpaintingObject