Adaptive Slimming for Scalable and Efficient Speech Enhancement
Speech enhancement (SE) enables robust speech recognition, real-time communication, hearing aids, and other applications where speech quality is crucial. However, deploying such systems on resource-constrained devices involves choosing a static trade-off between performance and computational efficiency. In this paper, we introduce dynamic slimming to DEMUCS, a popular SE architecture, making it scalable and input-adaptive. Slimming lets the model operate at different utilization factors (UF), each corresponding to a different performance/efficiency trade-off, effectively mimicking multiple model sizes without the extra storage costs. In addition, a router subnet, trained end-to-end with the backbone, determines the optimal UF for the current input. Thus, the system saves resources by adaptively selecting smaller UFs when additional complexity is unnecessary. We show that our solution is Pareto-optimal against individual UFs, confirming the benefits of dynamic routing. When training the proposed dynamically-slimmable model to use 10% of its capacity on average, we obtain the same or better speech quality as the equivalent static 25% utilization while reducing MACs by 29%.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational EfficiencySpeech RecognitionSpeech EnhancementSimilar Papers 제목 키워드 기반
DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming
Current multimodal large language models (MLLMs) face significant challenges in visual document understanding (VDU) tasks due to the high resolution, dense text, and complex layouts typical of document images. These char…
document understandingBLOOM-Net: Blockwise Optimization for Masking Networks Toward Scalable and Efficient Speech Enhancement
In this paper, we present a blockwise optimization method for masking-based networks (BLOOM-Net) for training scalable speech enhancement networks. Here, we design our network with a residual learning scheme and train th…
Speech EnhancementSpeech Enhancement using Adaptive Mean Median Deviation and EMD Technique
During the acquisition of the speech signal by the non-contact Speech Sensor (SS), the signal is degraded by severe colored noises which are non-linear and non-uniform in nature. Therefore, in this study, a new approach …
Speech EnhancementTrainable Adaptive Window Switching for Speech Enhancement
This study proposes a trainable adaptive window switching (AWS) method and apply it to a deep-neural-network (DNN) for speech enhancement in the modified discrete cosine transform domain. Time-frequency (T-F) mask proces…
Speech EnhancementHigh-Fidelity Speech Enhancement via Discrete Audio Tokens
Recent autoregressive transformer-based speech enhancement (SE) methods have shown promising results by leveraging advanced semantic understanding and contextual modeling of speech. However, these approaches often rely o…
Speech Enhancement