Speech-Driven Facial Reenactment Using Conditional Generative Adversarial Networks
We present a novel approach to generating photo-realistic images of a face with accurate lip sync, given an audio input. By using a recurrent neural network, we achieved mouth landmarks based on audio features. We exploited the power of conditional generative adversarial networks to produce highly-realistic face conditioned on a set of landmarks. These two networks together are capable of producing a sequence of natural faces in sync with an input audio track.
Code (0)
등록된 구현이 없습니다.
Tasks
Constrained Lip-synchronizationSimilar Papers 제목 키워드 기반
High-fidelity Facial Avatar Reconstruction from Monocular Video with Generative Priors
High-fidelity facial avatar reconstruction from a monocular video is a significant research problem in computer graphics and computer vision. Recently, Neural Radiance Field (NeRF) has shown impressive novel view renderi…
Face ReenactmentNeRFNovel View SynthesisVocal Bursts Intensity PredictionParametric Implicit Face Representation for Audio-Driven Facial Reenactment
Audio-driven facial reenactment is a crucial technique that has a range of applications in film-making, virtual avatars and video conferences. Existing works either employ explicit intermediate face representations (e.g.…
Data AugmentationImage GenerationImage InpaintingIMU2Face: Real-time Gesture-driven Facial Reenactment
We present IMU2Face, a gesture-driven facial reenactment system. To this end, we combine recent advances in facial motion capture and inertial measurement units (IMUs) to control the facial expressions of a person in a t…
Neural Voice Puppetry: Audio-driven Facial Reenactment
We present Neural Voice Puppetry, a novel approach for audio-driven facial video synthesis. Given an audio sequence of a source person or digital assistant, we generate a photo-realistic output video of a target person t…
Face ModelNeural RenderingTalking Face GenerationTalking Head Generation+2FACEGAN: Facial Attribute Controllable rEenactment GAN
The face reenactment is a popular facial animation method where the person's identity is taken from the source image and the facial motion from the driving image. Recent works have demonstrated high quality results by co…
AttributeFace Reenactment