paper-with-me

Papers

Taming Diffusion Probabilistic Models for Character Control

2024-04-23 · Rui Chen, Mingyi Shi, Shaoli Huang, Ping Tan, Taku Komura, Xuelin Chen

We present a novel character control framework that effectively utilizes motion diffusion probabilistic models to generate high-quality and diverse character animations, responding in real-time to a variety of dynamic user-supplied control signals. At the heart of our method lies a transformer-based Conditional Autoregressive Motion Diffusion Model (CAMDM), which takes as input the character's historical motion and can generate a range of diverse potential future motions conditioned on high-level, coarse user control. To meet the demands for diversity, controllability, and computational efficiency required by a real-time controller, we incorporate several key algorithmic designs. These include separate condition tokenization, classifier-free guidance on past motion, and heuristic future trajectory extension, all designed to address the challenges associated with taming motion diffusion probabilistic models for character control. As a result, our work represents the first model that enables real-time generation of high-quality, diverse character animations based on user interactive control, supporting animating the character in multiple styles with a single unified model. We evaluate our method on a diverse set of locomotion skills, demonstrating the merits of our method over existing character controllers. Project page and source codes: https://aiganimation.github.io/CAMDM/

📄 PDF Abstract BibTeX arXiv:2404.15121

Code (1)

AIGAnimation/CAMDM pytorch

Tasks

Computational EfficiencyDiversity

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

TITAN-Guide: Taming Inference-Time AligNment for Guided Text-to-Video Diffusion Models

2025-08-01 · Christian Simon, Masato Ishii, Akio Hayakawa, Zhi Zhong 외 arxiv

In the recent development of conditional diffusion models still require heavy supervised fine-tuning for performing control on a category of tasks. Training-free conditioning via guidance with off-the-shelf models is a f…

ChronosObserver: Taming 4D World with Hyperspace Diffusion Sampling

2025-12-01 · Qisen Wang, Yifan Zhao, Peisen Shen, Jialu Li 외 arxiv

Although prevailing camera-controlled video generation models can produce cinematic results, lifting them directly to the generation of 3D-consistent and high-fidelity time-synchronized multi-view videos remains challeng…

Data AugmentationVideo Generation

Pantheon360: Taming Digital Twin Generation via 3D-Aware 360° Video Diffusion

2026-05-25 · Ting-Hsuan Chen, Ying-Huan Chen, Tao Tu, Jie-Ying Lee 외 arxiv

Generating complete digital twins from videos requires precise camera control, global scene coverage, and strict spatial-temporal consistency constraints that remain challenging for perspective video generators due to th…

Scene GenerationVideo Generation

VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control

2024-07-17 · Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace 외

Modern text-to-video synthesis models demonstrate coherent, photorealistic generation of complex videos from a text description. However, most existing models lack fine-grained control over camera movement, which is crit…

Video Generation

D-Flow: Differentiating through Flows for Controlled Generation

2024-02-21 · Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer 외

Taming the generation outcome of state of the art Diffusion and Flow-Matching (FM) models without having to re-train a task-specific model unlocks a powerful tool for solving inverse problems, conditional generation, and…