Beat-Aligned Spectrogram-to-Sequence Generation of Rhythm-Game Charts

2023-11-22 · Jayeon Yi, Sungho Lee, Kyogu Lee

In the heart of "rhythm games" - games where players must perform actions in sync with a piece of music - are "charts", the directives to be given to players. We newly formulate chart generation as a sequence generation task and train a Transformer using a large dataset. We also introduce tempo-informed preprocessing and training procedures, some of which are suggested to be integral for a successful training. Our model is found to outperform the baselines on a large dataset, and is also found to benefit from pretraining and finetuning.

📄 PDF Abstract BibTeX arXiv:2311.13687

Code (0)

등록된 구현이 없습니다.

Tasks

Rhythm

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling

2021-07-05 · ACL 2021 5 · Lanqing Xue, Kaitao Song, Duocai Wu, Xu Tan 외

Rap generation, which aims to produce lyrics and corresponding singing beats, needs to model both rhymes and rhythms. Previous works for rap generation focused on rhyming lyrics but ignored rhythmic beats, which are impo…

Language Modeling Language Modelling Rhythm

Classification of Arrhythmia by Using Deep Learning with 2-D ECG Spectral Image Representation

2020-05-14 · Amin Ullah, Syed M. Anwar, Muhammad Bilal, Raja M Mehmood

The electrocardiogram (ECG) is one of the most extensively employed signals used in the diagnosis and prediction of cardiovascular diseases (CVDs). The ECG signals can capture the heart's rhythmic irregularities, commonl…

General Classification Specificity Time Series Time Series Analysis

MotionBeat: Motion-Aligned Music Representation via Embodied Contrastive Learning and Bar-Equivariant Contact-Aware Encoding

2025-10-15 · Xuanchen Wang, Heng Wang, Weidong Cai arxiv

Music is both an auditory and an embodied phenomenon, closely linked to human motion and naturally expressed through dance. However, most existing audio representations neglect this embodied dimension, limiting their abi…

Representation Learning Contrastive Learning Emotion Recognition Beat Tracking

JukeDrummer: Conditional Beat-aware Audio-domain Drum Accompaniment Generation via Transformer VQ-VAE

2022-10-12 · Yueh-Kao Wu, Ching-Yu Chiu, Yi-Hsuan Yang

This paper proposes a model that generates a drum track in the audio domain to play along to a user-provided drum-free recording. Specifically, using paired data of drumless tracks and the corresponding human-made drum t…

Beat Tracking Decoder

Drop the beat! Freestyler for Accompaniment Conditioned Rapping Voice Generation

2024-08-28 · Ziqian Ning, Shuai Wang, Yuepeng Jiang, Jixun Yao 외

Rap, a prominent genre of vocal performance, remains underexplored in vocal generation. General vocal synthesis depends on precise note and duration inputs, requiring users to have related musical knowledge, which limits…

Language Modeling Language Modelling