MagicStyle: Portrait Stylization Based on Reference Image

2024-09-12 · Zhaoli Deng, Kaibin Zhou, Fanyi Wang, Zhenpeng Mi

The development of diffusion models has significantly advanced the research on image stylization, particularly in the area of stylizing a content image based on a given style image, which has attracted many scholars. The main challenge in this reference image stylization task lies in how to maintain the details of the content image while incorporating the color and texture features of the style image. This challenge becomes even more pronounced when the content image is a portrait which has complex textural details. To address this challenge, we propose a diffusion model-based reference image stylization method specifically for portraits, called MagicStyle. MagicStyle consists of two phases: Content and Style DDIM Inversion (CSDI) and Feature Fusion Forward (FFF). The CSDI phase involves a reverse denoising process, where DDIM Inversion is performed separately on the content image and the style image, storing the self-attention query, key and value features of both images during the inversion process. The FFF phase executes forward denoising, harmoniously integrating the texture and color information from the pre-stored feature queries, keys and values into the diffusion generation process based on our Well-designed Feature Fusion Attention (FFA). We conducted comprehensive comparative and ablation experiments to validate the effectiveness of our proposed MagicStyle and FFA.

📄 PDF Abstract BibTeX arXiv:2409.08156

Code (0)

등록된 구현이 없습니다.

Tasks

DenoisingImage Stylization

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Attention 설명 없음

FFF A log-time alternative to feedforward layers outperforming both the vanilla feedforward and mixture-of-experts approaches.

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

A Framework for Portrait Stylization with Skin-Tone Awareness and Nudity Identification

2024-03-21 · Seungkwon Kim, Sangyeon Kim, Seung-Hun Nam

Portrait stylization is a challenging task involving the transformation of an input portrait image into a specific style while preserving its inherent characteristics. The recent introduction of Stable Diffusion (SD) has…

HyperStyle3D: Text-Guided 3D Portrait Stylization via Hypernetworks

2023-04-19 · Zhuo Chen, Xudong Xu, Yichao Yan, Ye Pan 외

Portrait stylization is a long-standing task enabling extensive applications. Although 2D-based methods have made great progress in recent years, real-world applications such as metaverse and games often demand 3D conten…

Attribute

Realtime Fewshot Portrait Stylization Based On Geometric Alignment

2022-11-28 · Xinrui Wang, Zhuoru Li, Xiao Zhou, Yusuke Iwasawa 외

This paper presents a portrait stylization method designed for real-time mobile applications with limited style examples available. Previous learning based stylization methods suffer from the geometric and semantic gaps …

Portrait Diffusion: Training-free Face Stylization with Chain-of-Painting

2023-12-03 · Jin Liu, Huaibo Huang, Chao Jin, Ran He

Face stylization refers to the transformation of a face into a specific portrait style. However, current methods require the use of example-based adaptation approaches to fine-tune pre-trained generative models so that t…

Image Reconstruction

Domain Generalizable Portrait Style Transfer

2025-07-06 · Xinbo Wang, Wenju Xu, Qing Zhang, Wei-Shi Zheng arxiv

This paper presents a portrait style transfer method that generalizes well to various different domains while enabling high-quality semantic-aligned stylization on regions including hair, eyes, eyelashes, skins, lips, an…

Semantic correspondence Style Transfer