Conditional Image Generation and Manipulation for User-Specified Content
In recent years, Generative Adversarial Networks (GANs) have improved steadily towards generating increasingly impressive real-world images. It is useful to steer the image generation process for purposes such as content creation. This can be done by conditioning the model on additional information. However, when conditioning on additional information, there still exists a large set of images that agree with a particular conditioning. This makes it unlikely that the generated image is exactly as envisioned by a user, which is problematic for practical content creation scenarios such as generating facial composites or stock photos. To solve this problem, we propose a single pipeline for text-to-image generation and manipulation. In the first part of our pipeline we introduce textStyleGAN, a model that is conditioned on text. In the second part of our pipeline we make use of the pre-trained weights of textStyleGAN to perform semantic facial image manipulation. The approach works by finding semantic directions in latent space. We show that this method can be used to manipulate facial images for a wide range of attributes. Finally, we introduce the CelebTD-HQ dataset, an extension to CelebA-HQ, consisting of faces and corresponding textual descriptions.
Code (3)
Tasks
Conditional Image GenerationImage GenerationImage ManipulationText to Image GenerationText-to-Image GenerationSimilar Papers 제목 키워드 기반
Towards Real-time Text-driven Image Manipulation with Unconditional Diffusion Models
Recent advances in diffusion models enable many powerful instruments for image editing. One of these instruments is text-driven image manipulations: editing semantic attributes of an image according to the provided text …
GPUImage ManipulationLGTM: Training-Free Light-Guided Text-to-Image Diffusion Model via Initial Noise Manipulation
Diffusion models have demonstrated high-quality performance in conditional text-to-image generation, particularly with structural cues such as edges, layouts, and depth. However, lighting conditions have received limited…
Text-to-Image GenerationIR-GAN: Image Manipulation with Linguistic Instruction by Increment Reasoning
Conditional image generation is an active research topic including text2image and image translation. Recently image manipulation with linguistic instruction brings new challenges of multimodal conditional generation. How…
Conditional Image GenerationGenerative Adversarial NetworkImage GenerationImage ManipulationMichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
Despite the recent success of face image generation with GANs, conditional hair editing remains challenging due to the under-explored complexity of its geometry and appearance. In this paper, we present MichiGAN (Multi-I…
Conditional Image GenerationImage GenerationConditional Generative Modeling for Images, 3D Animations, and Video
This dissertation attempts to drive innovation in the field of generative modeling for computer vision, by exploring novel formulations of conditional generative models, and innovative applications in images, 3D animatio…
ArticlesDecoderDenoisingGaussian Processes+2