DualSep: A Light-weight dual-encoder convolutional recurrent network for real-time in-car speech separation

2024-09-13 · Ziqian Wang, Jiayao Sun, Zihan Zhang, Xingchen Li, Jie Liu, Lei Xie

Advancements in deep learning and voice-activated technologies have driven the development of human-vehicle interaction. Distributed microphone arrays are widely used in in-car scenarios because they can accurately capture the voices of passengers from different speech zones. However, the increase in the number of audio channels, coupled with the limited computational resources and low latency requirements of in-car systems, presents challenges for in-car multi-channel speech separation. To migrate the problems, we propose a lightweight framework that cascades digital signal processing (DSP) and neural networks (NN). We utilize fixed beamforming (BF) to reduce computational costs and independent vector analysis (IVA) to provide spatial prior. We employ dual encoders for dual-branch modeling, with spatial encoder capturing spatial cues and spectral encoder preserving spectral information, facilitating spatial-spectral fusion. Our proposed system supports both streaming and non-streaming modes. Experimental results demonstrate the superiority of the proposed system across various metrics. With only 0.83M parameters and 0.39 real-time factor (RTF) on an Intel Core i7 (2.6GHz) CPU, it effectively separates speech into distinct speech zones. Our demos are available at https://honee-w.github.io/DualSep/.

📄 PDF Abstract BibTeX arXiv:2409.08610

Code (0)

등록된 구현이 없습니다.

Tasks

CPUSpeech Separation

CabinSep: IR-Augmented Mask-Based MVDR for Real-Time In-Car Speech Separation with Distributed Heterogeneous Arrays

2025-09-01 · Runduo Han, Yanxin Hu, Yihui Fu, Zihan Zhang 외 arxiv

Separating overlapping speech from multiple speakers is crucial for effective human-vehicle interaction. This paper proposes CabinSep, a lightweight neural mask-based minimum variance distortionless response (MVDR) speec…

Speech Recognition Speech Separation Data Augmentation

Blind microscopy image denoising with a deep residual and multiscale encoder/decoder network

2021-05-01 · Fabio Hernán Gil Zuluaga, Francesco Bardozzo, Jorge Iván Ríos Patiño, Roberto Tagliaferri

In computer-aided diagnosis (CAD) focused on microscopy, denoising improves the quality of image analysis. In general, the accuracy of this process may depend both on the experience of the microscopist and on the equipme…

Decoder Denoising Image Denoising Specificity+1

MFPNet: Multi-scale Feature Propagation Network For Lightweight Semantic Segmentation

2023-09-10 · Guoan Xu, Wenjing Jia, Tao Wu, Ligeng Chen

In contrast to the abundant research focusing on large-scale models, the progress in lightweight semantic segmentation appears to be advancing at a comparatively slower pace. However, existing compact methods often suffe…

Decoder Segmentation Semantic Segmentation

CLAP Convolutional Lightweight Autoencoder for Plant Disease Classification

2026-02-21 · Asish Bera, Subhajit Roy, Sudiptendu Banerjee arxiv

Convolutional neural networks have remarkably progressed the performance of distinguishing plant diseases, severity grading, and nutrition deficiency prediction using leaf images. However, these tasks become more challen…

DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks

2022-02-15 · Jiachen Zhong, Junying Chen, Ajmal Mian

CNN architectures are generally heavy on memory and computational requirements which makes them infeasible for embedded systems with limited hardware resources. We propose dual convolutional kernels (DualConv) for constr…

image-classification Image Classification object-detection Object Detection+1