Disentangled and Controllable Face Image Generation via 3D Imitative-Contrastive Learning
We propose DiscoFaceGAN, an approach for face image generation of virtual people with disentangled, precisely-controllable latent representations for identity of non-existing people, expression, pose, and illumination. We embed 3D priors into adversarial learning and train the network to imitate the image formation of an analytic 3D face deformation and rendering process. To deal with the generation freedom induced by the domain gap between real and rendered faces, we further introduce contrastive learning to promote disentanglement by comparing pairs of generated images. Experiments show that through our imitative-contrastive learning, the factor variations are very well disentangled and the properties of a generated face can be precisely controlled. We also analyze the learned latent space and present several meaningful properties supporting factor disentanglement. Our method can also be used to embed real images into the disentangled latent space. We hope our method could provide new understandings of the relationship between physical properties and deep image synthesis.
Code (4)
Tasks
Contrastive LearningDisentanglementImage GenerationSimilar Papers 제목 키워드 기반
Semi-Supervised StyleGAN for Disentanglement Learning
Disentanglement learning is crucial for obtaining disentangled representations and controllable generation. Current disentanglement methods face several inherent limitations: difficulty with high-resolution images, prima…
DisentanglementRepresentation LearningSketch2Human: Deep Human Generation with Disentangled Geometry and Appearance Control
Geometry- and appearance-controlled full-body human image generation is an interesting but challenging task. Existing solutions are either unconditional or dependent on coarse conditions (e.g., pose, text), thus lacking …
Face GenerationImage GenerationContent-style disentangled representation for controllable artistic image stylization and generation
Controllable artistic image stylization and generation aims to render the content provided by text or image with the learned artistic style, where content and style decoupling is the key to achieve satisfactory results. …
DisentanglementImage StylizationDisentangled GANs for Controllable Generation of High-Resolution Images
Generative adversarial networks (GANs) have achieved great success at generating realistic samples. However, achieving disentangled and controllable generation still remains challenging for GANs, especially in the high-r…
DisentanglementVocal Bursts Intensity PredictionDRDM: A Disentangled Representations Diffusion Model for Synthesizing Realistic Person Images
Person image synthesis with controllable body poses and appearances is an essential task owing to the practical needs in the context of virtual try-on, image editing and video production. However, existing methods face s…
Image GenerationPose TransferVirtual Try-on