LAMP: Language-Assisted Motion Planning for Controllable Video Generation

2025-12-03 · Muhammed Burak Kizil, Enes Sanli, Niloy J. Mitra, Erkut Erdem, Aykut Erdem, Duygu Ceylan arxiv

Video generation has achieved remarkable progress in visual fidelity and controllability, enabling conditioning on text, layout, or motion. Among these, motion control - specifying object dynamics and camera trajectories - is essential for composing complex, cinematic scenes, yet existing interfaces remain limited. We introduce LAMP that leverages large language models (LLMs) as motion planners to translate natural language descriptions into explicit 3D trajectories for dynamic objects and (relatively defined) cameras. LAMP defines a motion domain-specific language (DSL), inspired by cinematography conventions. By harnessing program synthesis capabilities of LLMs, LAMP generates structured motion programs from natural language, which are deterministically mapped to 3D trajectories. We construct a large-scale procedural dataset pairing natural text descriptions with corresponding motion programs and 3D trajectories. Experiments demonstrate LAMP's improved performance in motion controllability and alignment with user intent compared to state-of-the-art alternatives establishing the first framework for generating both object and camera motions directly from natural language specifications. Code, models and data are available on our project page.

📄 PDF Abstract BibTeX arXiv:2512.03619

Code (0)

등록된 구현이 없습니다.

Tasks

Program Synthesis Video Generation Motion Planning

LAMP: Long-Horizon Adaptive Manipulation Planning for Multi-Robot Collaboration in Cluttered Space

2026-06-28 · Shuai Zhou, Yorai Shaoul, Jiaoyang Li arxiv

Multi-robot manipulation requires jointly reasoning about contact formations, robot motions under coupled dynamics, and collision avoidance. Systematically searching over this large space is difficult and becomes increas…

Reinforcement Learning Collision Avoidance Robot Manipulation

LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioning

2024-10-09 · Zhe Li, Weihao Yuan, Yisheng He, Lingteng Qiu 외

Language plays a vital role in the realm of human motion. Existing methods have largely depended on CLIP text embeddings for motion generation, yet they fall short in effectively aligning language and motion due to CLIP'…

Large Language Model Motion Captioning Motion Generation Representation Learning+2

LAMP: Lane-Aligned Motion Primitives for Feasible Trajectory Prediction

2026-06-25 · Sangjin Han, Hoseong Jung, Jeongtae Her, Changhyun Choi 외 arxiv

Motion forecasting is essential for autonomous driving systems to enable safe decision-making and planning in complex driving scenarios. While existing predictors excel at minimizing standard displacement errors, they of…

Trajectory Prediction Autonomous Driving Motion Forecasting

LaMP: Learning Vision-Language-Action Policy with 3D Scene Flow as Latent Motion Prior

2026-03-26 · Xinkai Wang, Chenyi Wang, Yifu Xu, Mingzhe Ye 외 arxiv

We introduce \textbf{LaMP}, a dual-expert Vision-Language-Action framework that embeds dense 3D scene flow as a latent motion prior for robotic manipulation.Existing VLA models regress actions directly from 2D semantic v…

Reverse Constitutional AI: A Framework for Controllable Toxic Data Generation via Probability-Clamped RLAIF

2026-04-20 · Yuan Fang, Yiming Luo, Aimin Zhou, Fei Tan arxiv

Ensuring the safety of large language models (LLMs) requires robust red teaming, yet the systematic synthesis of high-quality toxic data remains under-explored. We propose Reverse Constitutional AI (R-CAI), a framework f…

Reinforcement Learning Red Teaming