VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Network
We present a novel high-fidelity real-time neural vocoder called VocGAN. A recently developed GAN-based vocoder, MelGAN, produces speech waveforms in real-time. However, it often produces a waveform that is insufficient in quality or inconsistent with acoustic characteristics of the input mel spectrogram. VocGAN is nearly as fast as MelGAN, but it significantly improves the quality and consistency of the output waveform. VocGAN applies a multi-scale waveform generator and a hierarchically-nested discriminator to learn multiple levels of acoustic properties in a balanced way. It also applies the joint conditional and unconditional objective, which has shown successful results in high-resolution image synthesis. In experiments, VocGAN synthesizes speech waveforms 416.7x faster on a GTX 1080Ti GPU and 3.24x faster on a CPU than real-time. Compared with MelGAN, it also exhibits significantly improved quality in multiple evaluation metrics including mean opinion score (MOS) with minimal additional overhead. Additionally, compared with Parallel WaveGAN, another recently developed high-fidelity vocoder, VocGAN is 6.98x faster on a CPU and exhibits higher MOS.
Code (2)
Tasks
CPUGPUSpeech SynthesisMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FlowVocoder: A small Footprint Neural Vocoder based Normalizing flow for Speech Synthesis
Recently, autoregressive neural vocoders have provided remarkable performance in generating high-fidelity speech and have been able to produce synthetic speech in real-time. However, autoregressive neural vocoders such a…
Density EstimationSpeech Synthesistext-to-speechText to SpeechSource-Filter HiFi-GAN: Fast and Pitch Controllable High-Fidelity Neural Vocoder
Our previous work, the unified source-filter GAN (uSFGAN) vocoder, introduced a novel architecture based on the source-filter theory into the parallel waveform generative adversarial network to achieve high voice quality…
CPUGenerative Adversarial NetworkWOLONet: Wave Outlooker for Efficient and High Fidelity Speech Synthesis
Recently, GAN-based neural vocoders such as Parallel WaveGAN, MelGAN, HiFiGAN, and UnivNet have become popular due to their lightweight and parallel structure, resulting in a real-time synthesized waveform with high fide…
CPUSpeech SynthesisVocal Bursts Intensity PredictionStyleMelGAN: An Efficient High-Fidelity Adversarial Vocoder with Temporal Adaptive Normalization
In recent years, neural vocoders have surpassed classical speech generation approaches in naturalness and perceptual quality of the synthesized speech. Computationally heavy models like WaveNet and WaveGlow achieve best …
Spectral Reconstructiontext-to-speechText to SpeechVocal Bursts Intensity PredictionVNet: A GAN-based Multi-Tier Discriminator Network for Speech Synthesis Vocoders
Since the introduction of Generative Adversarial Networks (GANs) in speech synthesis, remarkable achievements have been attained. In a thorough exploration of vocoders, it has been discovered that audio waveforms can be …
Speech Synthesis