Synthesizer Sound Matching Using Audio Spectrogram Transformers

2024-07-23 · Fred Bruford, Frederik Blang, Shahan Nercessian

Systems for synthesizer sound matching, which automatically set the parameters of a synthesizer to emulate an input sound, have the potential to make the process of synthesizer programming faster and easier for novice and experienced musicians alike, whilst also affording new means of interaction with synthesizers. Considering the enormous variety of synthesizers in the marketplace, and the complexity of many of them, general-purpose sound matching systems that function with minimal knowledge or prior assumptions about the underlying synthesis architecture are particularly desirable. With this in mind, we introduce a synthesizer sound matching model based on the Audio Spectrogram Transformer. We demonstrate the viability of this model by training on a large synthetic dataset of randomly generated samples from the popular Massive synthesizer. We show that this model can reconstruct parameters of samples generated from a set of 16 parameters, highlighting its improved fidelity relative to multi-layer perceptron and convolutional neural network baselines. We also provide audio examples demonstrating the out-of-domain model performance in emulating vocal imitations, and sounds from other synthesizers and musical instruments.

📄 PDF Abstract BibTeX arXiv:2407.16643

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

Position-Wise Feed-Forward Layer 설명 없음

InverSynth: Deep Estimation of Synthesizer Parameter Configurations from Audio Signals

2018-12-15 · Oren Barkan, David Tsiris, Ori Katz, Noam Koenigstein

Sound synthesis is a complex field that requires domain expertise. Manual tuning of synthesizer parameters to match a specific sound can be an exhaustive task, even for experienced sound engineers. In this paper, we intr…

Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations

2025-09-09 · Paolo Combes, Stefan Weinzierl, Klaus Obermayer arxiv

Deep learning appears as an appealing solution for Automatic Synthesizer Programming (ASP), which aims to assist musicians and sound designers in programming sound synthesizers. However, integrating software synthesizers…

DiffMoog: a Differentiable Modular Synthesizer for Sound Matching

2024-01-23 · Noy Uzrad, Oren Barkan, Almog Elharar, Shlomi Shvartzman 외

This paper presents DiffMoog - a differentiable modular synthesizer with a comprehensive set of modules typically found in commercial instruments. Being differentiable, it allows integration into neural networks, enablin…

Audio Synthesis

Effective Pre-Training of Audio Transformers for Sound Event Detection

2024-09-14 · Florian Schmid, Tobias Morocutti, Francesco Foscarin, Jan Schlüter 외

We propose a pre-training pipeline for audio spectrogram transformers for frame-level sound event detection tasks. On top of common pre-training steps, we add a meticulously designed training routine on AudioSet frame-le…

Data Augmentation Event Detection Knowledge Distillation Sound Event Detection

An investigation of the reconstruction capacity of stacked convolutional autoencoders for log-mel-spectrograms

2023-01-18 · Anastasia Natsiou, Luca Longo, Sean O'Leary

In audio processing applications, the generation of expressive sounds based on high-level representations demonstrates a high demand. These representations can be used to manipulate the timbre and influence the synthesis…

Audio Compression