Continuous Speech Separation with Recurrent Selective Attention Network
While permutation invariant training (PIT) based continuous speech separation (CSS) significantly improves the conversation transcription accuracy, it often suffers from speech leakages and failures in separation at "hot spot" regions because it has a fixed number of output channels. In this paper, we propose to apply recurrent selective attention network (RSAN) to CSS, which generates a variable number of output channels based on active speaker counting. In addition, we propose a novel block-wise dependency extension of RSAN by introducing dependencies between adjacent processing blocks in the CSS framework. It enables the network to utilize the separation results from the previous blocks to facilitate the current block processing. Experimental results on the LibriCSS dataset show that the RSAN-based CSS (RSAN-CSS) network consistently improves the speech recognition accuracy over PIT-based models. The proposed block-wise dependency modeling further boosts the performance of RSAN-CSS.
Code (0)
등록된 구현이 없습니다.
Tasks
speech-recognitionSpeech RecognitionSpeech SeparationSimilar Papers 제목 키워드 기반
Continuous Speech Separation with Conformer
Continuous speech separation plays a vital role in complicated speech related tasks such as conversation transcription. The separation model extracts a single speaker signal from a mixed speech. In this paper, we use tra…
Speech SeparationDual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation
Transformers have been the most successful architecture for various speech modeling tasks, including speech separation. However, the self-attention mechanism in transformers with quadratic complexity is inefficient in co…
MambaSpeech SeparationState Space ModelsMulti-channel Speech Separation Using Spatially Selective Deep Non-linear Filters
In a multi-channel separation task with multiple speakers, we aim to recover all individual speech signals from the mixture. In contrast to single-channel approaches, which rely on the different spectro-temporal characte…
Speech SeparationRTFS-Net: Recurrent Time-Frequency Modelling for Efficient Audio-Visual Speech Separation
Audio-visual speech separation methods aim to integrate different modalities to generate high-quality separated speech, thereby enhancing the performance of downstream tasks such as speech recognition. Most existing stat…
Audio-Visual Speech Recognitionspeech-recognitionSpeech RecognitionSpeech Separation+1IIANet: An Intra- and Inter-Modality Attention Network for Audio-Visual Speech Separation
Recent research has made significant progress in designing fusion modules for audio-visual speech separation. However, they predominantly focus on multi-modal fusion at a single temporal scale of auditory and visual feat…
Speech Separation