Conditional Sound Generation Using Neural Discrete Time-Frequency Representation Learning
Deep generative models have recently achieved impressive performance in speech and music synthesis. However, compared to the generation of those domain-specific sounds, generating general sounds (such as siren, gunshots) has received less attention, despite their wide applications. In previous work, the SampleRNN method was considered for sound generation in the time domain. However, SampleRNN is potentially limited in capturing long-range dependencies within sounds as it only back-propagates through a limited number of samples. In this work, we propose a method for generating sounds via neural discrete time-frequency representation learning, conditioned on sound classes. This offers an advantage in efficiently modelling long-range dependencies and retaining local fine-grained structures within sound clips. We evaluate our approach on the UrbanSound8K dataset, compared to SampleRNN, with the performance metrics measuring the quality and diversity of generated sounds. Experimental results show that our method offers comparable performance in quality and significantly better performance in diversity.
Code (1)
Tasks
DiversityMusic GenerationRepresentation LearningSpeech SynthesisSimilar Papers 제목 키워드 기반
Masked Conditional Neural Networks for Automatic Sound Events Recognition
Deep neural network architectures designed for application domains other than sound, especially image recognition, may not optimally harness the time-frequency representation when adapted to the sound recognition problem…
Masked Conditional Neural Networks for Environmental Sound Classification
The ConditionaL Neural Network (CLNN) exploits the nature of the temporal sequencing of the sound signal represented in a spectrogram, and its variant the Masked ConditionaL Neural Network (MCLNN) induces the network to …
ClassificationEnvironmental Sound ClassificationGeneral ClassificationSound ClassificationParallel Complex Diffusion for Scalable Time Series Generation
Diffusion models learn data distributions indirectly through denoising, making the difficulty of generative modeling closely tied to the dependency structure of data. For time series, strong temporal dependence forces th…
Computational EfficiencyFast frequency discrimination and phoneme recognition using a biomimetic membrane coupled to a neural network
In the human ear, the basilar membrane plays a central role in sound recognition. When excited by sound, this membrane responds with a frequency-dependent displacement pattern that is detected and identified by the audit…
Phoneme RecognitionRecognition of Acoustic Events Using Masked Conditional Neural Networks
Automatic feature extraction using neural networks has accomplished remarkable success for images, but for sound recognition, these models are usually modified to fit the nature of the multi-dimensional temporal represen…