RePoseDM: Recurrent Pose Alignment and Gradient Guidance for Pose Guided Image Synthesis

2023-10-24 · Anant Khandelwal

Pose-guided person image synthesis task requires re-rendering a reference image, which should have a photorealistic appearance and flawless pose transfer. Since person images are highly structured, existing approaches require dense connections for complex deformations and occlusions because these are generally handled through multi-level warping and masking in latent space. The feature maps generated by convolutional neural networks do not have equivariance, and hence multi-level warping is required to perform pose alignment. Inspired by the ability of the diffusion model to generate photorealistic images from the given conditional guidance, we propose recurrent pose alignment to provide pose-aligned texture features as conditional guidance. Due to the leakage of the source pose in conditional guidance, we propose gradient guidance from pose interaction fields, which output the distance from the valid pose manifold given a predicted pose as input. This helps in learning plausible pose transfer trajectories that result in photorealism and undistorted texture details. Extensive results on two large-scale benchmarks and a user study demonstrate the ability of our proposed approach to generate photorealistic pose transfer under challenging scenarios. Additionally, we demonstrate the efficiency of gradient guidance in pose-guided image generation on the HumanArt dataset with fine-tuned stable diffusion.

📄 PDF Abstract BibTeX arXiv:2310.16074

Code (0)

등록된 구현이 없습니다.

Tasks

Image GenerationPose-Guided Image GenerationPose Transfervalid

Methods 이 논문이 사용한 방법론

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Inference-Time Alignment for Diffusion Models via Variationally Stable Doob's Matching

2026-01-10 · Jinyuan Chang, Chenguang Duan, Yuling Jiao, Yi Xu 외 arxiv

Inference-time alignment for diffusion models aims to adapt a pre-trained reference diffusion model toward a target distribution without retraining the reference score network, thereby preserving the generative capacity …

Graph Guided Diffusion: Unified Guidance for Conditional Graph Generation

2025-05-26 · Victor M. Tenorio, Nicolas Zilberstein, Santiago Segarra, Antonio G. Marques

Diffusion models have emerged as powerful generative models for graph generation, yet their use for conditional graph generation remains a fundamental challenge. In particular, guiding diffusion models on graphs under ar…

Computational Efficiency Fairness Graph Generation Link Prediction

CoDe: Blockwise Control for Denoising Diffusion Models

2025-02-03 · Anuj Singh, Sayak Mukherjee, Ahmad Beirami, Hadi Jamali-Rad

Aligning diffusion models to downstream tasks often requires finetuning new models or gradient-based guidance at inference time to enable sampling from the reward-tilted posterior. In this work, we explore a simple infer…

Denoising Instruction Following

General and Efficient Steering of Diffusion Models

2026-02-11 · Qingsong Wang, Mikhail Belkin, Yusu Wang arxiv

Steering diffusion models toward conditions unseen during training typically requires either retraining with conditional inputs or per-step gradient computations, both of which incur substantial computational overhead. W…

Gradient-Free Classifier Guidance for Diffusion Model Sampling

2024-11-23 · Rahul Shenoy, Zhihong Pan, Kaushik Balakrishnan, Qisen Cheng 외

Image generation using diffusion models have demonstrated outstanding learning capabilities, effectively capturing the full distribution of the training dataset. They are known to generate wide variations in sampled imag…

Image Generation Text to Image Generation Text-to-Image Generation