Key-point Guided Deformable Image Manipulation Using Diffusion Model
In this paper, we introduce a Key-point-guided Diffusion probabilistic Model (KDM) that gains precise control over images by manipulating the object's key-point. We propose a two-stage generative model incorporating an optical flow map as an intermediate output. By doing so, a dense pixel-wise understanding of the semantic relation between the image and sparse key point is configured, leading to more realistic image generation. Additionally, the integration of optical flow helps regulate the inter-frame variance of sequential images, demonstrating an authentic sequential image generation. The KDM is evaluated with diverse key-point conditioned image synthesis tasks, including facial image generation, human pose synthesis, and echocardiography video prediction, demonstrating the KDM is proving consistency enhanced and photo-realistic images compared with state-of-the-art models.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationImage ManipulationOptical Flow EstimationVideo PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PinchBot: Long-Horizon Deformable Manipulation with Guided Diffusion Policy
Pottery creation is a complicated art form that requires dexterous, precise and delicate actions to slowly morph a block of clay to a meaningful, and often useful 3D goal shape. In this work, we aim to create a robotic s…
Deformable 3D Shape Diffusion Model
The Gaussian diffusion model, initially designed for image generation, has recently been adapted for 3D point cloud generation. However, these adaptations have not fully considered the intrinsic geometric characteristics…
Image GenerationmodelPoint Cloud GenerationD-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous Deformable Manipulation
Mastering dexterous robotic manipulation of deformable objects is vital for overcoming the limitations of parallel grippers in real-world applications. Current trajectory optimisation approaches often struggle to solve s…
Deformable Object ManipulationMovement Primitive Diffusion: Learning Gentle Robotic Manipulation of Deformable Objects
Policy learning in robot-assisted surgery (RAS) lacks data efficient and versatile methods that exhibit the desired motion quality for delicate surgical interventions. To this end, we introduce Movement Primitive Diffusi…
Imitation LearningMotion GenerationDeformPAM: Data-Efficient Learning for Long-horizon Deformable Object Manipulation via Preference-based Action Alignment
In recent years, imitation learning has made progress in the field of robotic manipulation. However, it still faces challenges when addressing complex long-horizon tasks with deformable objects, such as high-dimensional …
Deformable Object ManipulationImitation Learning