paper-with-me

홈 › Papers

SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesis

2024-12-28 · Wenkun He, Yun Liu, Ruitao Liu, Li Yi

Synthesizing realistic human-object interaction motions is a critical problem in VR/AR and human animation. Unlike the commonly studied scenarios involving a single human or hand interacting with one object, we address a more generic multi-body setting with arbitrary numbers of humans, hands, and objects. This complexity introduces significant challenges in synchronizing motions due to the high correlations and mutual influences among bodies. To address these challenges, we introduce SyncDiff, a novel method for multi-body interaction synthesis using a synchronized motion diffusion strategy. SyncDiff employs a single diffusion model to capture the joint distribution of multi-body motions. To enhance motion fidelity, we propose a frequency-domain motion decomposition scheme. Additionally, we introduce a new set of alignment scores to emphasize the synchronization of different body motions. SyncDiff jointly optimizes both data sample likelihood and alignment likelihood through an explicit synchronization strategy. Extensive experiments across four datasets with various multi-body configurations demonstrate the superiority of SyncDiff over existing state-of-the-art motion synthesis methods.

📄 PDF Abstract BibTeX arXiv:2412.20104

Code (0)

등록된 구현이 없습니다.

Tasks

Human AnimationHuman-Object Interaction DetectionMotion Synthesis

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions

2023-06-08 · NeurIPS 2023 11

The remarkable capabilities of pretrained image diffusion models have been utilized not only for generating fixed-size images but also for creating panoramas. However, naive stitching of multiple images often results in …

Conditional Image GenerationDenoisingImage Generation

AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising

2024-06-11 · Zigeng Chen, Xinyin Ma, Gongfan Fang, Zhenxiong Tan 외

Diffusion models have garnered significant interest from the community for their great generative ability across various applications. However, their typical multi-step sequential-denoising nature gives rise to high cumu…

Denoising

TempoSyncDiff: Distilled Temporally-Consistent Diffusion for Low-Latency Audio-Driven Talking Head Generation

2026-03-06 · Soumya Mazumdar, Vineet Kumar Rakesh arxiv

Diffusion models have recently advanced photorealistic human synthesis, although practical talking-head generation (THG) remains constrained by high inference latency, temporal instability such as flicker and identity dr…

Talking Head Generation

AsyncDiff: Asynchronous Timestep Conditioning for Enhanced Text-to-Image Diffusion Inference

2025-12-21 · Longhuan Xu, Feng Yin, Cunjian Chen arxiv

Text-to-image diffusion inference typically follows synchronized schedules, where the numerical integrator advances the latent state to the same timestep at which the denoiser is conditioned. We propose an asynchronous i…

TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation

2024-10-05 · Haiyang Liu, Xingchao Yang, Tomoya Akiyama, Yuantian Huang 외

We present TANGO, a framework for generating co-speech body-gesture videos. Given a few-minute, single-speaker reference video and target speech audio, TANGO produces high-fidelity videos with synchronized body gestures.…

cross-modal alignmentRetrievalvalid