A Closed-form Solution to Universal Style Transfer
Universal style transfer tries to explicitly minimize the losses in feature space, thus it does not require training on any pre-defined styles. It usually uses different layers of VGG network as the encoders and trains several decoders to invert the features into images. Therefore, the effect of style transfer is achieved by feature transform. Although plenty of methods have been proposed, a theoretical analysis of feature transform is still missing. In this paper, we first propose a novel interpretation by treating it as the optimal transport problem. Then, we demonstrate the relations of our formulation with former works like Adaptive Instance Normalization (AdaIN) and Whitening and Coloring Transform (WCT). Finally, we derive a closed-form solution named Optimal Style Transfer (OST) under our formulation by additionally considering the content loss of Gatys. Comparatively, our solution can preserve better structure and achieve visually pleasing results. It is simple yet effective and we demonstrate its advantages both quantitatively and qualitatively. Besides, we hope our theoretical analysis can inspire future works in neural style transfer. Code is available at https://github.com/lu-m13/OptimalStyleTransfer.
Code (3)
Tasks
FormStyle TransferMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Real-time Universal Style Transfer on High-resolution Images via Zero-channel Pruning
Extracting effective deep features to represent content and style information is the key to universal style transfer. Most existing algorithms use VGG19 as the feature extractor, which incurs a high computational cost an…
Style TransferUniversal Photorealistic Style Transfer: A Lightweight and Adaptive Approach
Photorealistic style transfer aims to apply stylization while preserving the realism and structure of input content. However, existing methods often encounter challenges such as color tone distortions, dependency on pair…
GPUStyle TransferSuper-ResolutionVideo Style TransferCollaborative Distillation for Ultra-Resolution Universal Style Transfer
Universal style transfer methods typically leverage rich representations from deep Convolutional Neural Network (CNN) models (e.g., VGG-19) pre-trained on large collections of images. Despite the effectiveness, its appli…
DecoderGPUKnowledge DistillationStyle TransferA Comprehensive Comparison between Neural Style Transfer and Universal Style Transfer
Style transfer aims to transfer arbitrary visual styles to content images. We explore algorithms adapted from two papers that try to solve the problem of style transfer while generalizing on unseen styles or compromised …
Style TransferCLIPGaussian: Universal and Multimodal Style Transfer Based on Gaussian Splatting
Gaussian Splatting (GS) has recently emerged as an efficient representation for rendering 3D scenes from 2D images and has been extended to images, videos, and dynamic 4D content. However, applying style transfer to GS-b…
Style Transfer