DreamCom: Finetuning Text-guided Inpainting Model for Image Composition
The goal of image composition is merging a foreground object into a background image to obtain a realistic composite image. Recently, generative composition methods are built on large pretrained diffusion models, due to their unprecedented image generation ability. However, they are weak in preserving the foreground object details. Inspired by recent text-to-image generation customized for certain object, we propose DreamCom by treating image composition as text-guided image inpainting customized for certain object. Specifically , we finetune pretrained text-guided image inpainting model based on a few reference images containing the same object, during which the text prompt contains a special token associated with this object. Then, given a new background, we can insert this object into the background with the text prompt containing the special token. In practice, the inserted object may be adversely affected by the background, so we propose masked attention mechanisms to avoid negative background interference. Experimental results on DreamEditBench and our contributed MureCom dataset show the outstanding performance of our DreamCom.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationImage InpaintingObjectText to Image GenerationText-to-Image GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DreamComposer: Controllable 3D Object Generation via Multi-View Conditions
Utilizing pre-trained 2D large-scale generative models, recent works are capable of generating high-quality novel views from a single in-the-wild image. However, due to the lack of information from multiple views, these …
3D Object ReconstructionNovel View SynthesisObjectObject ReconstructionDreamComposer++: Empowering Diffusion Models with Multi-View Conditions for 3D Content Generation
Recent advancements in leveraging pre-trained 2D diffusion models achieve the generation of high-quality novel views from a single in-the-wild image. However, existing works face challenges in producing controllable nove…
3D Object ReconstructionNovel View SynthesisImagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting
Text-guided image editing can have a transformative impact in supporting creative applications. A key challenge is to generate edits that are faithful to input text prompts, while consistent with input images. We present…
Image InpaintingObjecttext-guided-image-editingDAFT-GAN: Dual Affine Transformation Generative Adversarial Network for Text-Guided Image Inpainting
In recent years, there has been a significant focus on research related to text-guided image inpainting. However, the task remains challenging due to several constraints, such as ensuring alignment between the image and …
Generative Adversarial NetworkImage GenerationImage InpaintingText-Guided Neural Image Inpainting
Image inpainting task requires filling the corrupted image with contents coherent with the context. This research field has achieved promising progress by using neural image inpainting methods. Nevertheless, there is sti…
DescriptiveImage GenerationImage InpaintingImage-text matching+3