HS-Diffusion: Semantic-Mixing Diffusion for Head Swapping

2022-12-13 · Qinghe Wang, Lijie Liu, Miao Hua, Pengfei Zhu, WangMeng Zuo, QinGhua Hu, Huchuan Lu, Bing Cao

Image-based head swapping task aims to stitch a source head to another source body flawlessly. This seldom-studied task faces two major challenges: 1) Preserving the head and body from various sources while generating a seamless transition region. 2) No paired head swapping dataset and benchmark so far. In this paper, we propose a semantic-mixing diffusion model for head swapping (HS-Diffusion) which consists of a latent diffusion model (LDM) and a semantic layout generator. We blend the semantic layouts of source head and source body, and then inpaint the transition region by the semantic layout generator, achieving a coarse-grained head swapping. Semantic-mixing LDM can further implement a fine-grained head swapping with the inpainted layout as condition by a progressive fusion process, while preserving head and body with high-quality reconstruction. To this end, we propose a semantic calibration strategy for natural inpainting and a neck alignment for geometric realism. Importantly, we construct a new image-based head swapping benchmark and design two tailor-designed metrics (Mask-FID and Focal-FID). Extensive experiments demonstrate the superiority of our framework. The code will be available: https://github.com/qinghew/HS-Diffusion.

📄 PDF Abstract BibTeX arXiv:2212.06458

Code (1)

qinghew/hs-diffusion 공식 구현

Methods 이 논문이 사용한 방법론

Inpainting Train a convolutional neural network to generate the contents of an arbitrary image region conditioned on its surroundings.

Latent Diffusion Model Diffusion models applied to latent spaces, which are normally built with (Variational) Autoencoders.

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

DirectSwap: Mask-Free Cross-Identity Training and Benchmarking for Expression-Consistent Video Head Swapping

2025-12-10 · Yanan Wang, Shengcai Liao, Panwen Hu, Xin Li 외 arxiv

Video head swapping aims to replace the entire head of a video subject, including facial identity, head shape, and hairstyle, with that of a reference image, while preserving the target body, background, and motion dynam…

Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models

2024-09-11 · Sanoojan Baliah, Qinliang Lin, Shengcai Liao, Xiaodan Liang 외

Despite promising progress in face swapping task, realistic swapped images remain elusive, often marred by artifacts, particularly in scenarios involving high pose variation, color differences, and occlusion. To address …

Denoising Disentanglement Face Swapping

Safeguarding Facial Identity against Diffusion-based Face Swapping via Cascading Pathway Disruption

2026-01-21 · Liqin Wang, Qianyue Hu, Wei Lu, Xiangyang Luo arxiv

The rapid evolution of diffusion models has democratized face swapping but also raises concerns about privacy and identity security. Existing proactive defenses, often adapted from image editing attacks, prove ineffectiv…

Face Swapping Image Editing

High-Fidelity Diffusion Face Swapping with ID-Constrained Facial Conditioning

2025-03-28 · Dailan He, Xiahong Wang, Shulun Wang, Guanglu Song 외

Face swapping aims to seamlessly transfer a source facial identity onto a target while preserving target attributes such as pose and expression. Diffusion models, known for their superior generative capabilities, have re…

Attribute Face Swapping

DiffSwap: High-Fidelity and Controllable Face Swapping via 3D-Aware Masked Diffusion

2023-01-01 · CVPR 2023 1 · Wenliang Zhao, Yongming Rao, Weikang Shi, Zuyan Liu 외

In this paper, we propose DiffSwap, a diffusion model based framework for high-fidelity and controllable face swapping. Unlike previous work that relies on carefully designed network architectures and loss functions …

Face Swapping