No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Prompts
Diffusion models have recently advanced human motion generation, producing realistic and diverse animations from textual prompts. However, adapting these models to unseen actions or styles typically requires additional motion capture data and full retraining, which is costly and difficult to scale. We propose a post-training framework based on Reinforcement Learning that fine-tunes pretrained motion diffusion models using only textual prompts, without requiring any motion ground truth. Our approach employs a pretrained text-motion retrieval network as a reward signal and optimizes the diffusion policy with Denoising Diffusion Policy Optimization, effectively shifting the model's generative distribution toward the target domain without relying on paired motion data. We evaluate our method on cross-dataset adaptation and leave-one-out motion experiments using the HumanML3D and KIT-ML datasets across both latent- and joint-space diffusion architectures. Results from quantitative metrics and user studies show that our approach consistently improves the quality and diversity of generated motions, while preserving performance on the original distribution. Our approach is a flexible, data-efficient, and privacy-preserving solution for motion adaptation.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
StableMotion: Training Motion Cleanup Models with Unpaired Corrupted Data
Motion capture (mocap) data often exhibits visually jarring artifacts due to inaccurate sensors and post-processing. Cleaning this corrupted data can require substantial manual effort from human experts, which can be a c…
Motion GenerationSmoCap: Movement Reconstruction under Morphology-Pose Ambiguity through Unified Scale-Pose Canonicalization
Movement reconstruction pipelines need estimates that support interpretable joint motion and subject morphology, not only low marker fitting error. The same marker fitting error can be explained by different mixtures of …
Sign Language Motion Capture Dataset for Data-driven Synthesis
This paper presents a new 3D motion capture dataset of Czech Sign Language (CSE). Its main purpose is to provide the data for further analysis and data-based automatic synthesis of CSE utterances. The content of the data…
Physics-Guided Human Motion Capture with Pose Probability Modeling
Incorporating physics in human motion capture to avoid artifacts like floating, foot sliding, and ground penetration is a promising direction. Existing solutions always adopt kinematic results as reference motions, and t…
DenoisingAn end-to-end (deep) neural network applied to raw EEG, fNIRs and body motion data for data fusion and BCI classification task without any pre-/post-processing
Brain computer interfaces (BCI) using EEG, fNIRS and body motion (MoCap) data are getting more attention due to the fact that fNIRS and MoCap are not prone to movement artifacts similar to other brain imaging techniques …
Action DetectionActivity RecognitionEEGElectroencephalogram (EEG)+1