paper-with-me

Papers

Controllable Motion Generation via Diffusion Modal Coupling

2025-03-04 · Luobin Wang, Hongzhan Yu, Chenning Yu, Sicun Gao, Henrik Christensen

Diffusion models have recently gained significant attention in robotics due to their ability to generate multi-modal distributions of system states and behaviors. However, a key challenge remains: ensuring precise control over the generated outcomes without compromising realism. This is crucial for applications such as motion planning or trajectory forecasting, where adherence to physical constraints and task-specific objectives is essential. We propose a novel framework that enhances controllability in diffusion models by leveraging multi-modal prior distributions and enforcing strong modal coupling. This allows us to initiate the denoising process directly from distinct prior modes that correspond to different possible system behaviors, ensuring sampling to align with the training distribution. We evaluate our approach on motion prediction using the Waymo dataset and multi-task control in Maze2D environments. Experimental results show that our framework outperforms both guidance-based techniques and conditioned models with unimodal priors, achieving superior fidelity, diversity, and controllability, even in the absence of explicit conditioning. Overall, our approach provides a more reliable and scalable solution for controllable motion generation in robotics.

📄 PDF Abstract BibTeX arXiv:2503.02353

Code (1)

robinwangsd/diffusion-with-multi-modal-priors 공식 구현

Tasks

DenoisingDiversityMotion GenerationMotion Planningmotion predictionTrajectory Forecasting

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…
ALIGN In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss…

Similar Papers 제목 키워드 기반

Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models

2023-04-03 · Wenjie Yin, Ruibo Tu, Hang Yin, Danica Kragic 외

Data-driven and controllable human motion synthesis and prediction are active research areas with various applications in interactive media and social robotics. Challenges remain in these fields for generating diverse mo…

DecoderMotion Synthesis

MMGen: Unified Multi-modal Image Generation and Understanding in One Go

2025-03-26 · Jiepeng Wang, Zhaoqing Wang, Hao Pan, YuAn Liu 외

A unified diffusion framework for multi-modal generation and understanding has the transformative potential to achieve seamless and controllable image diffusion and other cross-modal tasks. In this paper, we introduce MM…

Image Generation

ADMC: Attention-based Diffusion Model for Missing Modalities Feature Completion

2025-07-08 · Wei zhang, Juan Chen, Yanbo J. Wang, En Zhu 외

Multimodal emotion and intent recognition is essential for automated human-computer interaction, It aims to analyze users' speech, text, and visual information to predict their emotions or intent. One of the significant …

Intent Recognition

Content-style disentangled representation for controllable artistic image stylization and generation

2024-12-19 · Ma Zhuoqi, Zhang Yixuan, You Zejun, Tian Long 외

Controllable artistic image stylization and generation aims to render the content provided by text or image with the learned artistic style, where content and style decoupling is the key to achieve satisfactory results. …

DisentanglementImage Stylization

Controllable Text-to-Motion Generation via Modular Body-Part Phase Control

2026-03-20 · Minyue Dai, Ke Fan, Anyi Rao, Jingbo Wang 외 arxiv

Text-to-motion (T2M) generation is becoming a practical tool for animation and interactive avatars. However, modifying specific body parts while maintaining overall motion coherence remains challenging. Existing methods …