paper-with-me

Papers

Generative Motion Infilling From Imprecisely Timed Keyframes

2025-03-02 · Purvi Goel, Haotian Zhang, C. Karen Liu, Kayvon Fatahalian

Keyframes are a standard representation for kinematic motion specification. Recent learned motion-inbetweening methods use keyframes as a way to control generative motion models, and are trained to generate life-like motion that matches the exact poses and timings of input keyframes. However, the quality of generated motion may degrade if the timing of these constraints is not perfectly consistent with the desired motion. Unfortunately, correctly specifying keyframe timings is a tedious and challenging task in practice. Our goal is to create a system that synthesizes high-quality motion from keyframes, even if keyframes are imprecisely timed. We present a method that allows constraints to be retimed as part of the generation process. Specifically, we introduce a novel model architecture that explicitly outputs a time-warping function to correct mistimed keyframes, and spatial residuals that add pose details. We demonstrate how our method can automatically turn approximately timed keyframe constraints into diverse, realistic motions with plausible timing and detailed submovements.

📄 PDF Abstract BibTeX arXiv:2503.01016

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Let's Think Frame by Frame with VIP: A Video Infilling and Prediction Dataset for Evaluating Video Chain-of-Thought

2023-05-23 · Vaishnavi Himakunthala, Andy Ouyang, Daniel Rose, Ryan He 외

Despite exciting recent results showing vision-language systems' capacity to reason about images using natural language, their capacity for video reasoning remains under-explored. We motivate framing video reasoning as t…

DescriptiveVideo Prediction

Less is More: Improving Motion Diffusion Models with Sparse Keyframes

2025-03-18 · Jinseok Bae, Inwoo Hwang, Young Yoon Lee, Ziyu Guo 외

Recent advances in motion diffusion models have led to remarkable progress in diverse motion generation tasks, including text-to-motion synthesis. However, existing approaches represent motions as dense frame sequences, …

Motion GenerationMotion Synthesis

Let Your Video Listen to Your Music!

2025-06-23 · Xinyu Zhang, Dong Gong, Zicheng Duan, Anton Van Den Hengel 외

Aligning the rhythm of visual motion in a video with a given music track is a practical need in multimedia production, yet remains an underexplored task in autonomous video editing. Effective alignment between motion and…

GPUMusic GenerationRhythmVideo Editing+1

SAGA: Stochastic Whole-Body Grasping with Contact

2021-12-19 · Yan Wu, Jiahao Wang, Yan Zhang, Siwei Zhang 외

The synthesis of human grasping has numerous applications including AR/VR, video games and robotics. While methods have been proposed to generate realistic hand-object interaction for object grasping and manipulation, th…

Object

Revealing Disocclusions in Temporal View Synthesis through Infilling Vector Prediction

2021-10-17 · Vijayalakshmi Kanchana, Nagabhushan Somraj, Suraj Yadwad, Rajiv Soundararajan

We consider the problem of temporal view synthesis, where the goal is to predict a future video frame from the past frames using knowledge of the depth and relative camera motion. In contrast to revealing the disoccluded…

Temporal View Synthesis