Multi-modal Conditional Bounding Box Regression for Music Score Following
This paper addresses the problem of sheet-image-based on-line audio-to-score alignment also known as score following. Drawing inspiration from object detection, a conditional neural network architecture is proposed that directly predicts x,y coordinates of the matching positions in a complete score sheet image at each point in time for a given musical performance. Experiments are conducted on a synthetic polyphonic piano benchmark dataset and the new method is compared to several existing approaches from the literature for sheet-image-based score following as well as an Optical Music Recognition baseline. The proposed approach achieves new state-of-the-art results and furthermore significantly improves the alignment performance on a set of real-world piano recordings by applying Impulse Responses as a data augmentation technique.
Code (1)
Tasks
Data Augmentationobject-detectionObject DetectionregressionSimilar Papers 제목 키워드 기반
Detecting Noteheads in Handwritten Scores with ConvNets and Bounding Box Regression
Noteheads are the interface between the written score and music. Each notehead on the page signifies one note to be played, and detecting noteheads is thus an unavoidable step for Optical Music Recognition. Noteheads are…
General ClassificationregressionBoundary Regression for Leitmotif Detection in Music Audio
Leitmotifs are musical phrases that are reprised in various forms throughout a piece. Due to diverse variations and instrumentation, detecting the occurrence of leitmotifs from audio recordings is a highly challenging ta…
Event Detectionobject-detectionObject DetectionregressionMulti-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics
This paper aims to test whether a multi-modal approach for music emotion recognition (MER) performs better than a uni-modal one on high-level song features and lyrics. We use 11 song features retrieved from the Spotify A…
Emotion RecognitionMusic Emotion RecognitionregressionMulti-Modal Pedestrian Detection with Large Misalignment Based on Modal-Wise Regression and Multi-Modal IoU
The combined use of multiple modalities enables accurate pedestrian detection under poor lighting conditions by using the high visibility areas from these modalities together. The vital assumption for the combination use…
Pedestrian DetectionregressionVideo2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model
Numerous studies in the field of music generation have demonstrated impressive performance, yet virtually no models are able to directly generate music to match accompanying videos. In this work, we develop a generative …
Music GenerationRhythm