Mask-Weighted Spatial Likelihood Coding for Speaker-Independent Joint Localization and Mask Estimation
Due to their robustness and flexibility, neural-driven beamformers are a popular choice for speech separation in challenging environments with a varying amount of simultaneous speakers alongside noise and reverberation. Time-frequency masks and relative directions of the speakers regarding a fixed spatial grid can be used to estimate the beamformer's parameters. To some degree, speaker-independence is achieved by ensuring a greater amount of spatial partitions than speech sources. In this work, we analyze how to encode both mask and positioning into such a grid to enable joint estimation of both quantities. We propose mask-weighted spatial likelihood coding and show that it achieves considerable performance in both tasks compared to baseline encodings optimized for either localization or mask estimation. In the same setup, we demonstrate superiority for joint estimation of both quantities. Conclusively, we propose a universal approach which can replace an upstream sound source localization system solely by adapting the training framework, making it highly relevant in performance-critical scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Sound Source LocalizationSpeech SeparationSimilar Papers 제목 키워드 기반
Incorporating Spatial Cues in Modular Speaker Diarization for Multi-channel Multi-party Meetings
Although fully end-to-end speaker diarization systems have made significant progress in recent years, modular systems often achieve superior results in real-world scenarios due to their greater adaptability and robustnes…
Clusteringspeaker-diarizationSpeaker DiarizationSpeech SeparationMCSAE: Masked Cross Self-Attentive Encoding for Speaker Embedding
In general, a self-attention mechanism has been applied for speaker embedding encoding. Previous studies focused on training the self-attention in a high-level layer, such as the last pooling layer. However, the effect o…
Speaker VerificationLearning Spatial Features from Audio-Visual Correspondence in Egocentric Videos
We propose a self-supervised method for learning representations based on spatial audio-visual correspondences in egocentric videos. Our method uses a masked auto-encoding framework to synthesize masked binaural (multi-c…
Active Speaker DetectionAudio DenoisingDenoisingAn improved uncertainty decoding scheme with weighted samples for DNN-HMM hybrid systems
In this paper, we advance a recently-proposed uncertainty decoding scheme for DNN-HMM (deep neural network - hidden Markov model) hybrid systems. This numerical sampling concept averages DNN outputs produced by a finite …
General ClassificationLearning Content-Weighted Deep Image Compression
Learning-based lossy image compression usually involves the joint optimization of rate-distortion performance. Most existing methods adopt spatially invariant bit length allocation and incorporate discrete entropy approx…
DecoderImage Compression