A Transformer-Based Visual Piano Transcription Algorithm

2024-11-13 · Uros Zivanovic, Carlos Eduardo Cancino-Chacón

Automatic music transcription (AMT) for musical performances is a long standing problem in the field of Music Information Retrieval (MIR). Visual piano transcription (VPT) is a multimodal subproblem of AMT which focuses on extracting a symbolic representation of a piano performance from visual information only (e.g., from a top-down video of the piano keyboard). Inspired by the success of Transformers for audio-based AMT, as well as their recent successes in other computer vision tasks, in this paper we present a Transformer based architecture for VPT. The proposed VPT system combines a piano bounding box detection model with an onset and pitch detection model, allowing our system to perform well in more naturalistic conditions like imperfect image crops around the piano and slightly tilted images.

📄 PDF Abstract BibTeX arXiv:2411.09037

Code (0)

등록된 구현이 없습니다.

Tasks

Information RetrievalMusic Information RetrievalMusic Transcription Retrieval

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Adam 설명 없음

Residual Connection 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Automatic Piano Transcription with Hierarchical Frequency-Time Transformer

2023-07-10 · Keisuke Toyama, Taketo Akama, Yukara Ikemiya, Yuhta Takida 외

Taking long-term spectral and temporal dependencies into account is essential for automatic piano transcription. This is especially helpful when determining the precise onset and offset for each note in the polyphonic pi…

Decoder Music Transcription

Exploring Transformer's potential on automatic piano transcription

2022-04-08 · Longshen Ou, Ziyi Guo, Emmanouil Benetos, Jiqing Han 외

Most recent research about automatic music transcription (AMT) uses convolutional neural networks and recurrent neural networks to model the mapping from music signals to symbolic notation. Based on a high-resolution pia…

Music Transcription

Multi-Task Multi-Frame Visual Piano Transcription

2026-08-04 · Yonghyun Kim, Hoyeol Sohn, Juhan Nam, Alexander Lerch arxiv

Audio-based piano transcription performs well on onset, pitch, and velocity, but the sustain pedal lets sound persist long after key release, so audio systems predict pedal-extended offsets rather than physical key relea…

A Data-Driven Analysis of Robust Automatic Piano Transcription

2024-02-02 · Drew Edwards, Simon Dixon, Emmanouil Benetos, Akira Maezawa 외

Algorithms for automatic piano transcription have improved dramatically in recent years due to new datasets and modeling techniques. Recent developments have focused primarily on adapting new neural network architectures…

Data Augmentation Music Transcription

AMT-APC: Automatic Piano Cover by Fine-Tuning an Automatic Music Transcription Model

2024-09-21 · Kazuma Komiya, Yoshihisa Fukuhara

There have been several studies on automatically generating piano covers, and recent advancements in deep learning have enabled the creation of more sophisticated covers. However, existing automatic piano cover models st…