Music- and Lyrics-driven Dance Synthesis
Lyrics often convey information about the songs that are beyond the auditory dimension, enriching the semantic meaning of movements and musical themes. Such insights are important in the dance choreography domain. However, most existing dance synthesis methods mainly focus on music-to-dance generation, without considering the semantic information. To complement it, we introduce JustLMD, a new multimodal dataset of 3D dance motion with music and lyrics. To the best of our knowledge, this is the first dataset with triplet information including dance motion, music, and lyrics. Additionally, we showcase a cross-modal diffusion-based network designed to generate 3D dance motion conditioned on music and lyrics. The proposed JustLMD dataset encompasses 4.6 hours of 3D dance motion in 1867 sequences, accompanied by musical tracks and their corresponding English lyrics.
Code (1)
Tasks
TripletMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
LM2D: Lyrics- and Music-Driven Dance Synthesis
Dance typically involves professional choreography with complex movements that follow a musical rhythm and can also be influenced by lyrical content. The integration of lyrics in addition to the auditory dimension, enric…
Motion GenerationPose EstimationRhythmEnabling Embodied Analogies in Intelligent Music Systems
The present methodology is aimed at cross-modal machine learning and uses multidisciplinary tools and methods drawn from a broad range of areas and disciplines, including music, systematic musicology, dance, motion captu…
Audio Signal ProcessingBIG-bench Machine LearningA Survey on Recent Deep Learning-driven Singing Voice Synthesis Systems
Singing voice synthesis (SVS) is a task that aims to generate audio signals according to musical scores and lyrics. With its multifaceted nature concerning music and language, producing singing voices indistinguishable f…
Deep LearningSinging Voice SynthesisLyrics Matter: Exploiting the Power of Learnt Representations for Music Popularity Prediction
Accurately predicting music popularity is a critical challenge in the music industry, offering benefits to artists, producers, and streaming platforms. Prior research has largely focused on audio features, social metadat…
Music-driven Dance Regeneration with Controllable Key Pose Constraints
In this paper, we propose a novel framework for music-driven dance motion synthesis with controllable key pose constraint. In contrast to methods that generate dance motion sequences only based on music without any other…
DecoderMotion Synthesis