DeepFilterNet2: Towards Real-Time Speech Enhancement on Embedded Devices for Full-Band Audio
Deep learning-based speech enhancement has seen huge improvements and recently also expanded to full band audio (48 kHz). However, many approaches have a rather high computational complexity and require big temporal buffers for real time usage e.g. due to temporal convolutions or attention. Both make those approaches not feasible on embedded devices. This work further extends DeepFilterNet, which exploits harmonic structure of speech allowing for efficient speech enhancement (SE). Several optimizations in the training procedure, data augmentation, and network structure result in state-of-the-art SE performance while reducing the real-time factor to 0.04 on a notebook Core-i5 CPU. This makes the algorithm applicable to run on embedded devices in real-time. The DeepFilterNet framework can be obtained under an open source license.
Code (1)
Tasks
CPUData AugmentationSpeech EnhancementSimilar Papers 제목 키워드 기반
DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement
Multi-frame algorithms for single-channel speech enhancement are able to take advantage from short-time correlations within the speech signal. Deep Filtering (DF) was proposed to directly estimate a complex filter in fre…
CPUSpeech EnhancementTraining DeepFilterNet with Accurate Room Acoustic Simulations Improves Single-Channel Speech Enhancement
We investigate how the realism of synthetic room impulse response (RIR) datasets affects the training of DeepFilterNet3 for single-channel speech enhancement. We compare a DNS4 image-source-method (ISM) RIR dataset with …
Speech RecognitionSpeech EnhancementDeepFilterNet: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering
Complex-valued processing has brought deep learning-based speech enhancement and signal extraction to a new level. Typically, the process is based on a time-frequency (TF) mask which is applied to a noisy spectrogram, wh…
Speech EnhancementHarmonic enhancement using learnable comb filter for light-weight full-band speech enhancement model
With fewer feature dimensions, filter banks are often used in light-weight full-band speech enhancement models. In order to further enhance the coarse speech in the sub-band domain, it is necessary to apply a post-filter…
RetrievalSpeech EnhancementOn real-time multi-stage speech enhancement systems
Recently, multi-stage systems have stood out among deep learning-based speech enhancement methods. However, these systems are always high in complexity, requiring millions of parameters and powerful computational resourc…
Speech Enhancement