PF-D2M: A Pose-free Diffusion Model for Universal Dance-to-Music Generation
Dance-to-music generation aims to generate music that is aligned with dance movements. Existing approaches typically rely on body motion features extracted from a single human dancer and limited dance-to-music datasets, which restrict their performance and applicability to real-world scenarios involving multiple dancers and non-human dancers. In this paper, we propose PF-D2M, a universal diffusion-based dance-to-music generation model that incorporates visual features extracted from dance videos. PF-D2M is trained with a progressive training strategy that effectively addresses data scarcity and generalization challenges. Both objective and subjective evaluations show that PF-D2M achieves state-of-the-art performance in dance-music alignment and music quality.
Code (0)
등록된 구현이 없습니다.
Tasks
Music GenerationSimilar Papers 제목 키워드 기반
Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion
We study the problem of symbolic music generation (e.g., generating piano rolls), with a technical focus on non-differentiable rule guidance. Musical rules are often expressed in symbolic form on note characteristics, su…
Music GenerationLongDanceDiff: Long-term Dance Generation with Conditional Diffusion Model
Dancing with music is always an essential human art form to express emotion. Due to the high temporal-spacial complexity, long-term 3D realist dance generation synchronized with music is challenging. Existing methods suf…
Motion GenerationGCDance: Genre-Controlled 3D Full Body Dance Generation Driven By Music
Generating high-quality full-body dance sequences from music is a challenging task as it requires strict adherence to genre-specific choreography. Moreover, the generated sequences must be both physically realistic and p…
RhythmMamba-Diffusion Model with Learnable Wavelet for Controllable Symbolic Music Generation
The recent surge in the popularity of diffusion models for image synthesis has attracted new attention to their potential for generation tasks in other domains. However, their applications to symbolic music generation re…
Image GenerationMambaMusic GenerationDiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
When hearing music, it is natural for people to dance to its rhythm. Automatic dance generation, however, is a challenging task due to the physical constraints of human motion and rhythmic alignment with target music. Co…
Representation LearningRhythmSuper-Resolution