Exploring Text-to-Motion Generation with Human Preference
This paper presents an exploration of preference learning in text-to-motion generation. We find that current improvements in text-to-motion generation still rely on datasets requiring expert labelers with motion capture systems. Instead, learning from human preference data does not require motion capture systems; a labeler with no expertise simply compares two generated motions. This is particularly efficient because evaluating the model's output is easier than gathering the motion that performs a desired task (e.g. backflip). To pioneer the exploration of this paradigm, we annotate 3,528 preference pairs generated by MotionGPT, marking the first effort to investigate various algorithms for learning from preference data. In particular, our exploration highlights important design choices when using preference data. Additionally, our experimental results show that preference learning has the potential to greatly improve current text-to-motion generative models. Our code and dataset are publicly available at https://github.com/THU-LYJ-Lab/InstructMotion}{https://github.com/THU-LYJ-Lab/InstructMotion to further facilitate research in this area.
Code (1)
Tasks
Motion GenerationSimilar Papers 제목 키워드 기반
MotionRL: Align Text-to-Motion Generation to Human Preferences with Multi-Reward Reinforcement Learning
We introduce MotionRL, the first approach to utilize Multi-Reward Reinforcement Learning (RL) for optimizing text-to-motion generation tasks and aligning them with human preferences. Previous works focused on improving n…
Motion Generationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought
Large Language Models (LLMs) have shown remarkable performance in various emotion recognition tasks, thereby piquing the research community's curiosity for exploring their potential in emotional intelligence. However, se…
Emotional IntelligenceEmotion RecognitionSentiment AnalysisMoDiPO: text-to-motion alignment via AI-feedback-driven Direct Preference Optimization
Diffusion Models have revolutionized the field of human motion generation by offering exceptional generation quality and fine-grained controllability through natural language conditioning. Their inherent stochasticity, t…
DiversityMotion GenerationMcSc: Motion-Corrective Preference Alignment for Video Generation with Self-Critic Hierarchical Reasoning
Text-to-video (T2V) generation has achieved remarkable progress in producing high-quality videos aligned with textual prompts. However, aligning synthesized videos with nuanced human preference remains challenging due to…
Reinforcement LearningVideo GenerationModel-Agnostic Human Preference Inversion in Diffusion Models
Efficient text-to-image generation remains a challenging task due to the high computational costs associated with the multi-step sampling in diffusion models. Although distillation of pre-trained diffusion models has bee…
Image GenerationmodelText to Image GenerationText-to-Image Generation