Ultra-Low Latency Speech Enhancement - A Comprehensive Study
Speech enhancement models should meet very low latency requirements typically smaller than 5 ms for hearing assistive devices. While various low-latency techniques have been proposed, comparing these methods in a controlled setup using DNNs remains blank. Previous papers have variations in task, training data, scripts, and evaluation settings, which make fair comparison impossible. Moreover, all methods are tested on small, simulated datasets, making it difficult to fairly assess their performance in real-world conditions, which could impact the reliability of scientific findings. To address these issues, we comprehensively investigate various low-latency techniques using consistent training on large-scale data and evaluate with more relevant metrics on real-world data. Specifically, we explore the effectiveness of asymmetric windows, learnable windows, adaptive time domain filterbanks, and the future-frame prediction technique. Additionally, we examine whether increasing the model size can compensate for the reduced window size, as well as the novel Mamba architecture in low-latency environments.
Code (0)
등록된 구현이 없습니다.
Tasks
MambaSpeech EnhancementMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fast-ULCNet: A fast and ultra low complexity network for single-channel speech enhancement
Single-channel speech enhancement algorithms are often used in resource-constrained embedded devices, where low latency and low complexity designs gain more importance. In recent years, researchers have proposed a wide v…
Speech EnhancementModulating State Space Model with SlowFast Framework for Compute-Efficient Ultra Low-Latency Speech Enhancement
Deep learning-based speech enhancement (SE) methods often face significant computational challenges when needing to meet low-latency requirements because of the increased number of frames to be processed. This paper intr…
Speech EnhancementLow bit rate binaural link for improved ultra low-latency low-complexity multichannel speech enhancement in Hearing Aids
Speech enhancement in hearing aids is a challenging task since the hardware limits the number of possible operations and the latency needs to be in the range of only a few milliseconds. We propose a deep-learning model c…
QuantizationSpeech EnhancementIncorporating Ultrasound Tongue Images for Audio-Visual Speech Enhancement through Knowledge Distillation
Audio-visual speech enhancement (AV-SE) aims to enhance degraded speech along with extra visual information such as lip videos, and has been shown to be more effective than audio-only speech enhancement. This paper propo…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Knowledge DistillationSpeech Enhancement+2Incorporating Ultrasound Tongue Images for Audio-Visual Speech Enhancement
Audio-visual speech enhancement (AV-SE) aims to enhance degraded speech along with extra visual information such as lip videos, and has been shown to be more effective than audio-only speech enhancement. This paper propo…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Knowledge DistillationSpeech Enhancement+2