DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability
In this paper we propose a novel generative approach, DiffRoll, to tackle automatic music transcription (AMT). Instead of treating AMT as a discriminative task in which the model is trained to convert spectrograms into piano rolls, we think of it as a conditional generative task where we train our model to generate realistic looking piano rolls from pure Gaussian noise conditioned on spectrograms. This new AMT formulation enables DiffRoll to transcribe, generate and even inpaint music. Due to the classifier-free nature, DiffRoll is also able to be trained on unpaired datasets where only piano rolls are available. Our experiments show that DiffRoll outperforms its discriminative counterpart by 19 percentage points (ppt.) and our ablation studies also indicate that it outperforms similar existing methods by 4.8 ppt. Source code and demonstration are available https://sony.github.io/DiffRoll/.
Code (0)
등록된 구현이 없습니다.
Tasks
Music TranscriptionSimilar Papers 제목 키워드 기반
Melody transcription via generative pre-training
Despite the central role that melody plays in music perception, it remains an open challenge in music information retrieval to reliably detect the notes of the melody present in an arbitrary music recording. A key challe…
Chord RecognitionInformation RetrievalMusic Information RetrievalRetrievalD3RM: A Discrete Denoising Diffusion Refinement Model for Piano Transcription
Diffusion models have been widely used in the generative domain due to their convincing performance in modeling complex data distributions. Moreover, they have shown competitive results on discriminative tasks, such as i…
DenoisingImage SegmentationMusic TranscriptionSemantic SegmentationUnsupervised Transcription of Piano Music
We present a new probabilistic model for transcribing piano music from audio to a symbolic form. Our model reflects the process by which discrete musical events give rise to acoustic signals that are then superimposed to…
A Hybrid Recurrent Neural Network For Music Transcription
We investigate the problem of incorporating higher-level symbolic score-like information into Automatic Music Transcription (AMT) systems to improve their performance. We use recurrent neural networks (RNNs) and their va…
Music TranscriptionNoise-to-Notes: Diffusion-based Generation and Refinement for Automatic Drum Transcription
Automatic drum transcription (ADT) is traditionally formulated as a discriminative task to predict drum events from audio spectrograms. In this work, we redefine ADT as a conditional generative task and introduce Noise-t…