LVNS-RAVE: Diversified audio generation with RAVE and Latent Vector Novelty Search
Evolutionary Algorithms and Generative Deep Learning have been two of the most powerful tools for sound generation tasks. However, they have limitations: Evolutionary Algorithms require complicated designs, posing challenges in control and achieving realistic sound generation. Generative Deep Learning models often copy from the dataset and lack creativity. In this paper, we propose LVNS-RAVE, a method to combine Evolutionary Algorithms and Generative Deep Learning to produce realistic and novel sounds. We use the RAVE model as the sound generator and the VGGish model as a novelty evaluator in the Latent Vector Novelty Search (LVNS) algorithm. The reported experiments show that the method can successfully generate diversified, novel audio samples under different mutation setups using different pre-trained RAVE models. The characteristics of the generation process can be easily controlled with the mutation parameters. The proposed algorithm can be a creative tool for sound artists and musicians.
Code (1)
Tasks
Audio GenerationDeep LearningEvolutionary AlgorithmsSimilar Papers 제목 키워드 기반
Conditional variational autoencoder to improve neural audio synthesis for polyphonic music sound
Deep generative models for audio synthesis have recently been significantly improved. However, the task of modeling raw-waveforms remains a difficult problem, especially for audio waveforms and music signals. Recently, t…
Audio SynthesisFrom General Vision to Reliable Traversability Estimation: Adapting Vision Foundation Models for Unstructured Outdoor Environments
Vision-based approaches have become the dominant paradigm for traversability estimation in unstructured outdoor environments, typically adapting vision foundation models (VFMs) via semantic segmentation supervision. Howe…
Semantic SegmentationDomain GeneralizationBRAVEn: Improving Self-Supervised Pre-training for Visual and Auditory Speech Recognition
Self-supervision has recently shown great promise for learning visual and auditory speech representations from unlabelled data. In this work, we propose BRAVEn, an extension to the recent RAVEn method, which learns speec…
speech-recognitionSpeech RecognitionDesigning Neural Synthesizers for Low-Latency Interaction
Neural Audio Synthesis (NAS) models offer interactive musical control over high-quality, expressive audio generators. While these models can operate in real-time, they often suffer from high latency, making them unsuitab…
Audio SynthesisBroaden Your Views for Self-Supervised Video Learning
Most successful self-supervised learning methods are trained to align the representations of two independent views from the data. State-of-the-art methods in video are inspired by image techniques, where these two views …
Audio ClassificationOptical Flow EstimationRepresentation LearningSelf-Supervised Action Recognition+2