Face Reconstruction from Voice using Generative Adversarial Networks
Voice profiling aims at inferring various human parameters from their speech, e.g. gender, age, etc. In this paper, we address the challenge posed by a subtask of voice profiling - reconstructing someone's face from their voice. The task is designed to answer the question: given an audio clip spoken by an unseen person, can we picture a face that has as many common elements, or associations as possible with the speaker, in terms of identity? To address this problem, we propose a simple but effective computational framework based on generative adversarial networks (GANs). The network learns to generate faces from voices by matching the identities of generated faces to those of the speakers, on a training set. We evaluate the performance of the network by leveraging a closely related task - cross-modal matching. The results show that our model is able to generate faces that match several biometric characteristics of the speaker, and results in matching accuracies that are much better than chance. The code is publicly available in https://github.com/cmu-mlsp/reconstructing_faces_from_voices
Code (1)
Tasks
Face ReconstructionSimilar Papers 제목 키워드 기반
Voice Aging with Audio-Visual Style Transfer
Face aging techniques have used generative adversarial networks (GANs) and style transfer learning to transform one's appearance to look younger/older. Identity is maintained by conditioning these generative networks on …
Style TransferTransfer LearningSingGAN: Generative Adversarial Network For High-Fidelity Singing Voice Generation
Deep generative models have achieved significant progress in speech synthesis to date, while high-fidelity singing voice synthesis is still an open problem for its long continuous pronunciation, rich high-frequency parts…
Generative Adversarial NetworkGPUSinging Voice SynthesisSpeech Synthesis+3Reconstructing faces from voices
Voice profiling aims at inferring various human parameters from their speech, e.g. gender, age, etc. In this paper, we address the challenge posed by a subtask of voice profiling - reconstructing someone's face from thei…
Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GAN
Singing voice synthesis (SVS) is the computer production of a human-like singing voice from given musical scores. To accomplish end-to-end SVS effectively and efficiently, this work adopts the acoustic model-neural vocod…
DenoisingGenerative Adversarial NetworkSinging Voice SynthesisRobust One-Shot Singing Voice Conversion
Recent progress in deep generative models has improved the quality of voice conversion in the speech domain. However, high-quality singing voice conversion (SVC) of unseen singers remains challenging due to the wider var…
Voice Conversion