Lifting 2D StyleGAN for 3D-Aware Face Generation
We propose a framework, called LiftedGAN, that disentangles and lifts a pre-trained StyleGAN2 for 3D-aware face generation. Our model is "3D-aware" in the sense that it is able to (1) disentangle the latent space of StyleGAN2 into texture, shape, viewpoint, lighting and (2) generate 3D components for rendering synthetic images. Unlike most previous methods, our method is completely self-supervised, i.e. it neither requires any manual annotation nor 3DMM model for training. Instead, it learns to generate images as well as their 3D components by distilling the prior knowledge in StyleGAN2 with a differentiable renderer. The proposed model is able to output both the 3D shape and texture, allowing explicit pose and lighting control over generated images. Qualitative and quantitative results show the superiority of our approach over existing methods on 3D-controllable GANs in content controllability while generating realistic high quality images.
Code (1)
Tasks
Face GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Face Generation and Editing with StyleGAN: A Survey
Our goal with this survey is to provide an overview of the state of the art deep learning methods for face generation and editing using StyleGAN. The survey covers the evolution of StyleGAN, from PGGAN to StyleGAN3, and …
Deep LearningFace GenerationFace SwappingStyle Transfer+1Grasping the Arrow of Time from the Singularity: Decoding Micromotion in Low-dimensional Latent Spaces from StyleGAN
The disentanglement of StyleGAN latent space has paved the way for realistic and controllable image editing, but does StyleGAN know anything about temporal motion, as it was only trained on static images? To study the mo…
DisentanglementFace GenerationDrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGAN
The research topic of sketch-to-portrait generation has witnessed a boost of progress with deep learning techniques. The recently proposed StyleGAN architectures achieve state-of-the-art generation ability but the origin…
Image GenerationExploring StyleGAN Latent Space for Face Alignment with Limited Training Data
With deep learning models growing in size over the years, sometimes exceeding a billion parameters now, the need for large, annotated training datasets grows too. To alleviate this problem, the interest in self-supervise…
Face AlignmentImage GenerationSelf-Supervised LearningTackling fake images in cybersecurity -- Interpretation of a StyleGAN and lifting its black-box
In today's digital age, concerns about the dangers of AI-generated images are increasingly common. One powerful tool in this domain is StyleGAN (style-based generative adversarial networks), a generative adversarial netw…