A StyleMap-Based Generator for Real-Time Image Projection and Local Editing
Generative adversarial networks (GANs) have been successful in synthesizing and manipulating synthetic but realistic images from latent vectors. However, it is still challenging for GANs to manipulate real images, especially in real-time. State-of-the-art GAN-based methods for editing real images suffer from time-consuming operations in projecting real images to latent vectors. Alternatively, an encoder can be trained to embed real images to the latent space instantly, but it loses details drastically. We propose StyleMapGAN, which adopts a novel representation of latent space, called stylemap, incorporating spatial dimensions into embedding. Because each spatial location in the stylemap contributes to its corresponding region of the generated images, the real-time projection through the encoder becomes accurate as well as editing real images becomes spatially controllable. Experimental results demonstrate that our method significantly outperforms state-of-the-art models in various image manipulation tasks such as local editing and image interpolation. Especially, detailed comparisons show that our local editing method successfully reflects not only the color and texture but also the shape of a reference image while preserving untargeted regions.
Code (0)
등록된 구현이 없습니다.
Tasks
Image ManipulationSimilar Papers 제목 키워드 기반
Exploiting Spatial Dimensions of Latent in GAN for Real-time Image Editing
Generative adversarial networks (GANs) synthesize realistic images from random latent vectors. Although manipulating the latent vectors controls the synthesized outputs, editing real images with GANs suffers from i) time…
Image ManipulationvalidSynthesizing 3D Shapes from Silhouette Image Collections using Multi-projection Generative Adversarial Networks
We present a new weakly supervised learning-based method for generating novel category-specific 3D shapes from unoccluded image collections. Our method is weakly supervised and only requires silhouette annotations from u…
Generative Adversarial NetworkWeakly-supervised LearningUVTomo-GAN: An adversarial learning based approach for unknown view X-ray tomographic reconstruction
Tomographic reconstruction recovers an unknown image given its projections from different angles. State-of-the-art methods addressing this problem assume the angles associated with the projections are known a-priori. Giv…
Improving Inversion and Generation Diversity in StyleGAN using a Gaussianized Latent Space
Modern Generative Adversarial Networks are capable of creating artificial, photorealistic images from latent vectors living in a low-dimensional learned latent space. It has been shown that a wide range of images can be …
DiversityUnsupervised Projection Networks for Generative Adversarial Networks
We propose the use of unsupervised learning to train projection networks that project onto the latent space of an already trained generator. We apply our method to a trained StyleGAN, and use our projection network to pe…
ClusteringImage Super-ResolutionSuper-Resolution