HiFi-Stream: Streaming Speech Enhancement with Generative Adversarial Networks
Speech Enhancement techniques have become core technologies in mobile devices and voice software simplifying downstream speech tasks. Still, modern Deep Learning (DL) solutions often require high amount of computational resources what makes their usage on low-resource devices challenging. We present HiFi-Stream, an optimized version of recently published HiFi++ model. Our experiments demonstrate that HiFiStream saves most of the qualities of the original model despite its size and computational complexity: the lightest version has only around 490k parameters which is 3.5x reduction in comparison to the original HiFi++ making it one of the smallest and fastest models available. The model is evaluated in streaming setting where it demonstrates its superior performance in comparison to modern baselines.
Code (1)
Tasks
Speech EnhancementSimilar Papers 제목 키워드 기반
Real-Time Streaming Mel Vocoding with Generative Flow Matching
The task of Mel vocoding, i.e., the inversion of a Mel magnitude spectrogram to an audio waveform, is still a key component in many text-to-speech (TTS) systems today. Based on generative flow matching, our prior work on…
Real-Time Streamable Generative Speech Restoration with Flow Matching
Diffusion-based generative models have greatly impacted the speech processing field in recent years, exhibiting high speech naturalness and spawning a new research direction. Their application in real-time communication …
Bandwidth ExtensionSpeech EnhancementHiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement
Generative adversarial networks have recently demonstrated outstanding performance in neural vocoding outperforming best autoregressive and flow-based models. In this paper, we show that this success can be extended to o…
Audio GenerationBandwidth ExtensionSpeech EnhancementIterative autoregression: a novel trick to improve your low-latency speech enhancement model
Streaming models are an essential component of real-time speech enhancement tools. The streaming regime constrains speech enhancement models to use only a tiny context of future information. As a result, the low-latency …
Speech EnhancementRT-LA-VocE: Real-Time Low-SNR Audio-Visual Speech Enhancement
In this paper, we aim to generate clean speech frame by frame from a live video stream and a noisy audio stream without relying on future inputs. To this end, we propose RT-LA-VocE, which completely re-designs every comp…
Speech Enhancement