From Audio Encoders to Piano Judges: Benchmarking Performance Understanding for Solo Piano
Our study investigates an approach for understanding musical performances through the lens of audio encoding models, focusing on the domain of solo Western classical piano music. Compared to composition-level attribute understanding such as key or genre, we identify a knowledge gap in performance-level music understanding, and address three critical tasks: expertise ranking, difficulty estimation, and piano technique detection, introducing a comprehensive Pianism-Labelling Dataset (PLD) for this purpose. We leverage pre-trained audio encoders, specifically Jukebox, Audio-MAE, MERT, and DAC, demonstrating varied capabilities in tackling downstream tasks, to explore whether domain-specific fine-tuning enhances capability in capturing performance nuances. Our best approach achieved 93.6\% accuracy in expertise ranking, 33.7\% in difficulty estimation, and 46.7\% in technique detection, with Audio-MAE as the overall most effective encoder. Finally, we conducted a case study on Chopin Piano Competition data using trained models for expertise ranking, which highlights the challenge of accurately assessing top-tier performances.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeBenchmarkingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PianoVAM: A Multimodal Piano Performance Dataset
The multimodal nature of music performance has driven increasing interest in data beyond the audio domain within the music information retrieval (MIR) community. This paper introduces PianoVAM, a comprehensive piano perf…
Information RetrievalHand Pose EstimationGenerative Modelling for Controllable Audio Synthesis of Expressive Piano Performance
We present a controllable neural audio synthesizer based on Gaussian Mixture Variational Autoencoders (GM-VAE), which can generate realistic piano performances in the audio domain that closely follows temporal conditions…
Audio SynthesisPiano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders
Automatic Music Transcription (AMT), aiming to get musical notes from raw audio, typically uses frame-level systems with piano-roll outputs or language model (LM)-based systems with note-level predictions. However, frame…
DecoderLanguage ModelingLanguage ModellingMusic TranscriptionSpiking Music: Audio Compression with Event Based Auto-encoders
Neurons in the brain communicate information via punctual events called spikes. The timing of spikes is thought to carry rich information, but it is not clear how to leverage this in digital systems. We demonstrate that …
Audio CompressionMusic CompressionPIAST: A Multimodal Piano Dataset with Audio, Symbolic and Text
While piano music has become a significant area of study in Music Information Retrieval (MIR), there is a notable lack of datasets for piano solo music with text labels. To address this gap, we present PIAST (PIano datas…
Information RetrievalMusic Information RetrievalMusic TaggingRetrieval+1