Exploring Multi-Modal Control in Music-Driven Dance Generation
Existing music-driven 3D dance generation methods mainly concentrate on high-quality dance generation, but lack sufficient control during the generation process. To address these issues, we propose a unified framework capable of generating high-quality dance movements and supporting multi-modal control, including genre control, semantic control, and spatial control. First, we decouple the dance generation network from the dance control network, thereby avoiding the degradation in dance quality when adding additional control information. Second, we design specific control strategies for different control information and integrate them into a unified framework. Experimental results show that the proposed dance generation framework outperforms state-of-the-art methods in terms of motion quality and controllability.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Vis2Mus: Exploring Multimodal Representation Mapping for Controllable Music Generation
In this study, we explore the representation mapping from the domain of visual arts to the domain of music, with which we can use visual arts as an effective handle to control music generation. Unlike most studies in mul…
Music GenerationRepresentation LearningStyle TransferExploring the Efficacy of Pre-trained Checkpoints in Text-to-Music Generation Task
Benefiting from large-scale datasets and pre-trained models, the field of generative models has recently gained significant momentum. However, most datasets for symbolic music are very small, which potentially limits the…
Music GenerationText-to-Music GenerationMusic2P: A Multi-Modal AI-Driven Tool for Simplifying Album Cover Design
In today's music industry, album cover design is as crucial as the music itself, reflecting the artist's vision and brand. However, many AI-driven album cover services require subscriptions or technical expertise, limiti…
Code GenerationImage SegmentationSemantic SegmentationMusic-driven Dance Regeneration with Controllable Key Pose Constraints
In this paper, we propose a novel framework for music-driven dance motion synthesis with controllable key pose constraint. In contrast to methods that generate dance motion sequences only based on music without any other…
DecoderMotion SynthesisEvery Image Listens, Every Image Dances: Music-Driven Image Animation
Image animation has become a promising area in multimodal research, with a focus on generating videos from reference images. While prior work has largely emphasized generic video generation guided by text, music-driven d…
Image AnimationVideo Generation