Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer

2026-03-19 · Chenyang Gu, Mingyuan Zhang, Haozhe Xie, Zhongang Cai, Lei Yang, Ziwei Liu arxiv

Prior motion generation largely follows two paradigms: continuous diffusion models that excel at kinematic control, and discrete token-based generators that are effective for semantic conditioning. To combine their strengths, we propose a three-stage framework comprising condition feature extraction (Perception), discrete token generation (Planning), and diffusion-based motion synthesis (Control). Central to this framework is MoTok, a diffusion-based discrete motion tokenizer that decouples semantic abstraction from fine-grained reconstruction by delegating motion recovery to a diffusion decoder, enabling compact single-layer tokens while preserving motion fidelity. For kinematic conditions, coarse constraints guide token generation during planning, while fine-grained constraints are enforced during control through diffusion-based optimization. This design prevents kinematic details from disrupting semantic token planning. On HumanML3D, our method significantly improves controllability and fidelity over MaskControl while using only one-sixth of the tokens, reducing trajectory error from 0.72 cm to 0.08 cm and FID from 0.083 to 0.029. Unlike prior methods that degrade under stronger kinematic constraints, ours improves fidelity, reducing FID from 0.033 to 0.014.

📄 PDF Abstract BibTeX arXiv:2603.19227

Code (0)

등록된 구현이 없습니다.

Tasks

Motion Synthesis

Results from the Paper

Rank	Task	Dataset	Model	Metrics
#31	Motion Synthesis	HumanML3D	Bridging Semantic and Kinematic Conditio	FID: 0.72

LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformers

2026-03-02 · Fabian Schmidt, Karol Fedurko, Markus Enzweiler, Abhinav Valada arxiv

While multimodal large language models (MLLMs) provide advanced reasoning for autonomous driving, translating their discrete semantic knowledge into continuous trajectories remains a fundamental challenge. Existing metho…

Autonomous Driving

EgoMotion: Hierarchical Reasoning and Diffusion for Egocentric Vision-Language Motion Generation

2026-04-21 · Ruibing Hou, Mingyue Zhou, Yuwei Gui, Mingshuang Luo 외 arxiv

Faithfully modeling human behavior in dynamic environments is a foundational challenge for embodied intelligence. While conditional motion synthesis has achieved significant advances, egocentric motion generation remains…

Motion Synthesis

DICArt: Advancing Category-level Articulated Object Pose Estimation in Discrete State-Spaces

2026-02-23 · Li Zhang, Mingyu Mei, Ailing Wang, Xianhui Meng 외 arxiv

Articulated object pose estimation is a core task in embodied AI. Existing methods typically regress poses in a continuous space, but often struggle with 1) navigating a large, complex search space and 2) failing to inco…

6D Pose Estimation

Bridging the Discrete-Continuous Gap: Unified Multimodal Generation via Coupled Manifold Discrete Absorbing Diffusion

2026-01-07 · Yuanfeng Xu, Yuhao Chen, Liang Lin, Guangrun Wang arxiv

The bifurcation of generative modeling into autoregressive approaches for discrete data (text) and diffusion approaches for continuous data (images) hinders the development of truly unified multimodal systems. While Mask…

multimodal generation Image Generation

Semantic-Aware Prefix Learning for Token-Efficient Image Generation

2026-03-26 · Qingfeng Li, Haoxian Zhang, Xu He, Songlin Tang 외 arxiv

Visual tokenizers play a central role in latent image generation by bridging high-dimensional images and tractable generative modeling. However, most existing tokenizers are still trained with reconstruction-dominated ob…

Representation Learning Image Generation