DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment
Video-driven neural face reenactment aims to synthesize realistic facial images that successfully preserve the identity and appearance of a source face, while transferring the target head pose and facial expressions. Existing GAN-based methods suffer from either distortions and visual artifacts or poor reconstruction quality, i.e., the background and several important appearance details, such as hair style/color, glasses and accessories, are not faithfully reconstructed. Recent advances in Diffusion Probabilistic Models (DPMs) enable the generation of high-quality realistic images. To this end, in this paper we present DiffusionAct, a novel method that leverages the photo-realistic image generation of diffusion models to perform neural face reenactment. Specifically, we propose to control the semantic space of a Diffusion Autoencoder (DiffAE), in order to edit the facial pose of the input images, defined as the head pose orientation and the facial expressions. Our method allows one-shot, self, and cross-subject reenactment, without requiring subject-specific fine-tuning. We compare against state-of-the-art GAN-, StyleGAN2-, and diffusion-based methods, showing better or on-par reenactment performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Face ReenactmentImage GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation
Controllable face generation poses critical challenges in generative modeling due to the intricate balance required between semantic controllability and photorealism. While existing approaches struggle with disentangling…
Zero-shot GeneralizationDiffusion Autoencoders for Few-shot Image Generation in Hyperbolic Space
Few-shot image generation aims to generate diverse and high-quality images for an unseen class given only a few examples in that class. However, existing methods often suffer from a trade-off between image quality and di…
DiversityImage GenerationA Tilted Seesaw: Revisiting Autoencoder Trade-off for Controllable Diffusion
In latent diffusion models, the autoencoder (AE) is typically expected to balance two capabilities: faithful reconstruction and a generation-friendly latent space (e.g., low gFID). In recent ImageNet-scale AE studies, we…
DisControlFace: Adding Disentangled Control to Diffusion Autoencoder for One-shot Explicit Facial Image Editing
In this work, we focus on exploring explicit fine-grained control of generative facial image editing, all while generating faithful facial appearances and consistent semantic details, which however, is quite challenging …
A Standard Processing Pipeline for High-accuracy Measurement of Few-shot Regression on Laser Induced Breakdown Spectroscopy
Laser-induced breakdown spectroscopy (LIBS) faces challenges in high-accuracy quantitative measurement under few-shot scenarios due to spectral noise and data scarcity. Traditional preprocessing methods often fail to pre…
Dimensionality ReductionData Augmentation