Two-Step Sound Source Separation: Training on Learned Latent Targets
In this paper, we propose a two-step training procedure for source separation via a deep neural network. In the first step we learn a transform (and it's inverse) to a latent space where masking-based separation performance using oracles is optimal. For the second step, we train a separation module that operates on the previously learned space. In order to do so, we also make use of a scale-invariant signal to distortion ratio (SI-SDR) loss function that works in the latent space, and we prove that it lower-bounds the SI-SDR in the time domain. We run various sound separation experiments that show how this approach can obtain better performance as compared to systems that learn the transform and the separation module jointly. The proposed methodology is general enough to be applicable to a large class of neural network end-to-end separation systems.
Code (2)
Tasks
Speech SeparationVocal Bursts Valence PredictionSimilar Papers 제목 키워드 기반
Improving Universal Sound Separation Using Sound Classification
Deep learning approaches have recently achieved impressive performance on both audio source separation and sound classification. Most audio source separation approaches focus only on separating sources belonging to a res…
Audio Source SeparationClassificationGeneral ClassificationSound ClassificationWeakly-supervised Audio-visual Sound Source Detection and Separation
Learning how to localize and separate individual object sounds in the audio channel of the video is a difficult task. Current state-of-the-art methods predict audio masks from artificially mixed spectrograms, known as Mi…
Audio Source SeparationDenoisingObjectSegmentation+3Co-Separating Sounds of Visual Objects
Learning how objects sound from video is challenging, since they often heavily overlap in a single audio channel. Current methods for visually-guided audio source separation sidestep the issue by training with artificial…
Audio DenoisingAudio Source SeparationDenoisingUnsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures using Spatial Information
We present a monophonic source separation system that is trained by only observing mixtures with no ground truth separation information. We use a deep clustering approach which trains on multi-channel mixtures and learns…
ClusteringDeep ClusteringMulti-Speaker Source SeparationSpeech SeparationOne-shot conditional audio filtering of arbitrary sounds
We consider the problem of separating a particular sound source from a single-channel mixture, based on only a short sample of the target source. Using SoundFilter, a wave-to-wave neural network architecture, we can trai…