Audio Source Separation Using Variational Autoencoders and Weak Class Supervision
In this paper, we propose a source separation method that is trained by observing the mixtures and the class labels of the sources present in the mixture without any access to isolated sources. Since our method does not require source class labels for every time-frequency bin but only a single label for each source constituting the mixture signal, we call this scenario as weak class supervision. We associate a variational autoencoder (VAE) with each source class within a non-negative (compositional) model. Each VAE provides a prior model to identify the signal from its associated class in a sound mixture. After training the model on mixtures, we obtain a generative model for each source class and demonstrate our method on one-second mixtures of utterances of digits from 0 to 9. We show that the separation performance obtained by source class supervision is as good as the performance obtained by source signal supervision.
Code (1)
Tasks
Audio Source SeparationDenoisingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Mixture of Dynamical Variational Autoencoders for Multi-Source Trajectory Modeling and Separation
In this paper, we propose a latent-variable generative model called mixture of dynamical variational autoencoders (MixDVAE) to model the dynamics of a system composed of multiple moving sources. A DVAE model is pre-train…
Audio Source SeparationMulti-Object TrackingObject TrackingTrajectory ModelingSource Separation of Multi-source Raw Music using a Residual Quantized Variational Autoencoder
I developed a neural audio codec model based on the residual quantized variational autoencoder architecture. I train the model on the Slakh2100 dataset, a standard dataset for musical source separation, composed of multi…
Zero-shot Audio Source Separation through Query-based Learningfrom Weakly-labeled Data
Deep learning techniques for separating audio into different sound sources face several challenges. Standard architectures require training separate models for different types of audio sources. Although some universal se…
Audio Source SeparationEvent DetectionSound Event DetectionZero-shot GeneralizationZero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data
Deep learning techniques for separating audio into different sound sources face several challenges. Standard architectures require training separate models for different types of audio sources. Although some universal se…
Audio Source SeparationAudio TaggingEvent DetectionSound Event Detection+1Weakly-supervised Audio-visual Sound Source Detection and Separation
Learning how to localize and separate individual object sounds in the audio channel of the video is a difficult task. Current state-of-the-art methods predict audio masks from artificially mixed spectrograms, known as Mi…
Audio Source SeparationDenoisingObjectSegmentation+3