DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations
The diffusion-based text-to-image model harbors immense potential in transferring reference style. However, current encoder-based approaches significantly impair the text controllability of text-to-image models while transferring styles. In this paper, we introduce DEADiff to address this issue using the following two strategies: 1) a mechanism to decouple the style and semantics of reference images. The decoupled feature representations are first extracted by Q-Formers which are instructed by different text descriptions. Then they are injected into mutually exclusive subsets of cross-attention layers for better disentanglement. 2) A non-reconstructive learning method. The Q-Formers are trained using paired images rather than the identical target, in which the reference image and the ground-truth image are with the same style or semantics. We show that DEADiff attains the best visual stylization results and optimal balance between the text controllability inherent in the text-to-image model and style similarity to the reference image, as demonstrated both quantitatively and qualitatively. Our project page is https://tianhao-qi.github.io/DEADiff/.
Code (2)
Tasks
DisentanglementSimilar Papers 제목 키워드 기반
Content-style disentangled representation for controllable artistic image stylization and generation
Controllable artistic image stylization and generation aims to render the content provided by text or image with the learned artistic style, where content and style decoupling is the key to achieve satisfactory results. …
DisentanglementImage StylizationGaussianBlender: Instant Stylization of 3D Gaussians with Disentangled Latent Spaces
3D stylization is central to game development, virtual reality, and digital arts, where the demand for diverse assets calls for scalable methods that support fast, high-fidelity manipulation. Existing text-to-3D stylizat…
PARASOL: Parametric Style Control for Diffusion Image Synthesis
We propose PARASOL, a multi-modal synthesis model that enables disentangled, parametric control of the visual style of the image by jointly conditioning synthesis on both content and a fine-grained visual style embedding…
Image GenerationJigsaw3D: Disentangled 3D Style Transfer via Patch Shuffling and Masking
Controllable 3D style transfer seeks to restyle a 3D asset so that its textures match a reference image while preserving the integrity and multi-view consistency. The prevalent methods either rely on direct reference sty…
Style TransferStyleFusion360: View-Consistent Head Stylization via Adaptive Style Modulation
3D head stylization enables expressive reimagining of human faces for creative visual experiences in digital media. Existing 3D-aware methods often require computationally intensive optimization or per-style fine-tuning,…