paper-with-me

홈 › Papers

D2Styler: Advancing Arbitrary Style Transfer with Discrete Diffusion Methods

2024-08-07 · Onkar Susladkar, Gayatri Deshmukh, Sparsh Mittal, Parth Shastri

In image processing, one of the most challenging tasks is to render an image's semantic meaning using a variety of artistic approaches. Existing techniques for arbitrary style transfer (AST) frequently experience mode-collapse, over-stylization, or under-stylization due to a disparity between the style and content images. We propose a novel framework called D$^2$Styler (Discrete Diffusion Styler) that leverages the discrete representational capability of VQ-GANs and the advantages of discrete diffusion, including stable training and avoidance of mode collapse. Our method uses Adaptive Instance Normalization (AdaIN) features as a context guide for the reverse diffusion process. This makes it easy to move features from the style image to the content image without bias. The proposed method substantially enhances the visual quality of style-transferred images, allowing the combination of content and style in a visually appealing manner. We take style images from the WikiArt dataset and content images from the COCO dataset. Experimental results demonstrate that D$^2$Styler produces high-quality style-transferred images and outperforms twelve existing methods on nearly all the metrics. The qualitative results and ablation studies provide further insights into the efficacy of our technique. The code is available at https://github.com/Onkarsus13/D2Styler.

📄 PDF Abstract BibTeX arXiv:2408.03558

Code (1)

onkarsus13/d2styler 공식 구현 jax

Tasks

Style Transfer

Methods 이 논문이 사용한 방법론

Instance Normalization Instance Normalization (also known as contrast normalization) is a normalization layer where: $$ y_{tijk} = \frac{x_{tijk} - \mu_{ti}}{\sqrt{\sigma_{ti}^2 +…
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…
Adaptive Instance Normalization 설명 없음

Similar Papers 제목 키워드 기반

RLMiniStyler: Light-weight RL Style Agent for Arbitrary Sequential Neural Style Generation

2025-05-07 · Jing Hu, Chengming Feng, Shu Hu, Ming-Ching Chang 외

Arbitrary style transfer aims to apply the style of any given artistic image to another content image. Still, existing deep learning-based methods often require significant computational costs to generate diverse stylize…

Multi-Task Learningreinforcement-learningReinforcement LearningStyle Transfer

CLIP3Dstyler: Language Guided 3D Arbitrary Neural Style Transfer

2023-05-25 · Ming Gao, Yanwu Xu, Yang Zhao, Tingbo Hou 외

In this paper, we propose a novel language-guided 3D arbitrary neural style transfer method (CLIP3Dstyler). We aim at stylizing any 3D scene with an arbitrary style from a text description, and synthesizing the novel sty…

Style Transfer

FastCLIPstyler: Optimisation-free Text-based Image Style Transfer Using Style Representations

2022-10-07 · Ananda Padhmanabhan Suresh, Sanjana Jain, Pavit Noinongyao, Ankush Ganguly 외

In recent years, language-driven artistic style transfer has emerged as a new type of style transfer technique, eliminating the need for a reference style image by using natural language descriptions of the style. The fi…

Style Transfer

StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fields

2023-03-19 · CVPR 2023 1 · Kunhao Liu, Fangneng Zhan, YiWen Chen, Jiahui Zhang 외

3D style transfer aims to render stylized novel views of a 3D scene with multi-view consistency. However, most existing work suffers from a three-way dilemma over accurate geometry reconstruction, high-quality stylizatio…

Style Transfer

DiffStyler: Diffusion-based Localized Image Style Transfer

2024-03-27 · Shaoxu Li

Image style transfer aims to imbue digital imagery with the distinctive attributes of style targets, such as colors, brushstrokes, shapes, whilst concurrently preserving the semantic integrity of the content. Despite the…

DenoisingStyle Transfer