TF-Mamba: A Time-Frequency Network for Sound Source Localization
Sound source localization (SSL) determines the position of sound sources using multi-channel audio data. It is commonly used to improve speech enhancement and separation. Extracting spatial features is crucial for SSL, especially in challenging acoustic environments. Recently, a novel structure referred to as Mamba demonstrated notable performance across various sequence-based modalities. This study introduces the Mamba for SSL tasks. We consider the Mamba-based model to analyze spatial features from speech signals by fusing both time and frequency features, and we develop an SSL system called TF-Mamba. This system integrates time and frequency fusion, with Bidirectional Mamba managing both time-wise and frequency-wise processing. We conduct the experiments on the simulated and real datasets. Experiments show that TF-Mamba significantly outperforms other advanced methods. The code will be publicly released in due course.
Code (0)
등록된 구현이 없습니다.
Tasks
MambaSound Source LocalizationSpeech EnhancementMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Stereo sound event localization and detection based on PSELDnet pretraining and BiMamba sequence modeling
Pre-training methods have achieved significant performance improvements in sound event localization and detection (SELD) tasks, but existing Transformer-based models suffer from high computational complexity. In this wor…
DecoderMambaSound Event Localization and DetectionDeFT-Mamba: Universal Multichannel Sound Separation and Polyphonic Audio Classification
This paper presents a framework for universal sound separation and polyphonic audio classification, addressing the challenges of separating and classifying individual sound sources in a multichannel mixture. The proposed…
Audio ClassificationClassificationMambaSELD-Mamba: Selective State-Space Model for Sound Event Localization and Detection with Source Distance Estimation
In the Sound Event Localization and Detection (SELD) task, Transformer-based models have demonstrated impressive capabilities. However, the quadratic complexity of the Transformer's self-attention mechanism results in co…
Computational EfficiencyEvent DetectionMambaSound Event Detection+1DCASE 2021 Task 3: Spectrotemporally-aligned Features for Polyphonic Sound Event Localization and Detection
Sound event localization and detection consists of two subtasks which are sound event detection and direction-of-arrival estimation. While sound event detection mainly relies on time-frequency patterns to distinguish dif…
Audio ClassificationDirection of Arrival EstimationEvent DetectionSound Event Detection+1Sound Source Localization for Human-Robot Interaction in Outdoor Environments
This paper presents a sound source localization strategy that relies on a microphone array embedded in an unmanned ground vehicle and an asynchronous close-talking microphone near the operator. A signal coarse alignment …
Sound Source Localization