Reference Channel Selection by Multi-Channel Masking for End-to-End Multi-Channel Speech Enhancement
In end-to-end multi-channel speech enhancement, the traditional approach of designating one microphone signal as the reference for processing may not always yield optimal results. The limitation is particularly in scenarios with large distributed microphone arrays with varying speaker-to-microphone distances or compact, highly directional microphone arrays where speaker or microphone positions change over time. Current mask-based methods often fix the reference channel during training, which makes it not possible to adaptively select the reference channel for optimal performance. To address this problem, we introduce an adaptive approach for selecting the optimal reference channel. Our method leverages a multi-channel masking-based scheme, where multiple masked signals are combined to generate a single-channel output signal. This enhanced signal is then used for loss calculation, while the reference clean speech is adjusted based on the highest scale-invariant signal-to-distortion ratio (SI-SDR). The experimental results on the Spear challenge simulated dataset D4 demonstrate the superiority of our proposed method over the conventional approach of using a fixed reference channel with single-channel masking.
Code (0)
등록된 구현이 없습니다.
Tasks
channel selectionSpeech EnhancementSimilar Papers 제목 키워드 기반
Multi-Channel Masking with Learnable Filterbank for Sound Source Separation
This work proposes a learnable filterbank based on a multi-channel masking framework for multi-channel source separation. The learnable filterbank is a 1D Conv layer, which transforms the raw waveform into a 2D represent…
Low Complexity Reconfigurable Modified FRM Architecture with Full Spectral Utilization for Efficient Channelizers
This paper proposes a design of low complexity, reconfigurable and narrow transition band (NTB) filter bank (FB). In our proposed Modified Frequency Response Masking (ModFRM) architecture, the modal filter and complement…
Learning Signal Representations for EEG Cross-Subject Channel Selection and Trial Classification
EEG technology finds applications in several domains. Currently, most EEG systems require subjects to wear several electrodes on the scalp to be effective. However, several channels might include noisy information, redun…
channel selectionEEGEeg DecodingElectroencephalogram (EEG)Spatial-frequency channels, shape bias, and adversarial robustness
What spatial frequency information do humans and neural networks use to recognize objects? In neuroscience, critical band masking is an established tool that can reveal the frequency-selective filters used for object rec…
EfficientSync: Real-Time Lip Synchronization via Deformation-Based Reference Texture Mixing
Audio-driven lip synchronization manipulates the mouth region of a talking-face video to match the driving audio while preserving head pose, identity, and background. Although the task is inherently local editing, prevai…