LayoutDiffuse: Adapting Foundational Diffusion Models for Layout-to-Image Generation
Layout-to-image generation refers to the task of synthesizing photo-realistic images based on semantic layouts. In this paper, we propose LayoutDiffuse that adapts a foundational diffusion model pretrained on large-scale image or text-image datasets for layout-to-image generation. By adopting a novel neural adaptor based on layout attention and task-aware prompts, our method trains efficiently, generates images with both high perceptual quality and layout alignment, and needs less data. Experiments on three datasets show that our method significantly outperforms other 10 generative models based on GANs, VQ-VAE, and diffusion models.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationLayout-to-Image GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Layered Rendering Diffusion Model for Controllable Zero-Shot Image Synthesis
This paper introduces innovative solutions to enhance spatial controllability in diffusion models reliant on text queries. We first introduce vision guidance as a foundational spatial cue within the perturbed distributio…
DenoisingImage GenerationLayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation
Recently, diffusion models have achieved great success in image synthesis. However, when it comes to the layout-to-image generation where an image often has a complex scene of multiple objects, how to make strong control…
Image GenerationLayout-to-Image GenerationObjectSynCity 3000: Bootstrapping Scene-Scale 3D Diffusion
We present SynCity 3000, a framework for generating 3D scenes that are globally coherent while enabling fine-grained layout control. Building on the ability of current image-to-3D generators to produce complex 3D assets …
Scene GenerationReason out Your Layout: Evoking the Layout Master from Large Language Models for Text-to-Image Synthesis
Recent advancements in text-to-image (T2I) generative models have shown remarkable capabilities in producing diverse and imaginative visuals based on text prompts. Despite the advancement, these diffusion models sometime…
Image GenerationVisual Layout Composer: Image-Vector Dual Diffusion Model for Design Layout Generation
This paper proposes an image-vector dual diffusion model for generative layout design. Distinct from prior efforts that mostly ignore element-level visual information our approach integrates the power of a pre-traine…
Layout DesignLayout GenerationPosition