MulSMo: Multimodal Stylized Motion Generation by Bidirectional Control Flow
Generating motion sequences conforming to a target style while adhering to the given content prompts requires accommodating both the content and style. In existing methods, the information usually only flows from style to content, which may cause conflict between the style and content, harming the integration. Differently, in this work we build a bidirectional control flow between the style and the content, also adjusting the style towards the content, in which case the style-content collision is alleviated and the dynamics of the style is better preserved in the integration. Moreover, we extend the stylized motion generation from one modality, i.e. the style motion, to multiple modalities including texts and images through contrastive learning, leading to flexible style control on the motion generation. Extensive experiments demonstrate that our method significantly outperforms previous methods across different datasets, while also enabling multimodal signals control. The code of our method will be made publicly available.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningMotion GenerationSimilar Papers 제목 키워드 기반
SMooGPT: Stylized Motion Generation using Large Language Models
Stylized motion generation is actively studied in computer graphics, especially benefiting from the rapid advances in diffusion models. The goal of this task is to produce a novel motion respecting both the motion conten…
Style TransferSMooDi: Stylized Motion Diffusion Model
We introduce a novel Stylized Motion Diffusion model, dubbed SMooDi, to generate stylized motion driven by content texts and style motion sequences. Unlike existing methods that either generate motion of various content …
modelMotion GenerationRSMT: Real-time Stylized Motion Transition for Characters
Styled online in-between motion generation has important application scenarios in computer animation and games. Its core challenge lies in the need to satisfy four critical requirements simultaneously: generation speed, …
DiversityMotion GenerationStyGazeTalk: Learning Stylized Generation of Gaze and Head Dynamics
Gaze and head movements play a central role in expressive 3D media, human-agent interaction, and immersive communication. Existing works often model facial components in isolation and lack mechanisms for generating perso…
EmoVid: A Multimodal Emotion Video Dataset for Emotion-Centric Video Understanding and Generation
Emotion plays a pivotal role in video-based expression, but existing video generation systems predominantly focus on low-level visual metrics while neglecting affective dimensions. Although emotion analysis has made prog…
Video Generation