Characterizing and Improving Stability in Neural Style Transfer
Recent progress in style transfer on images has focused on improving the quality of stylized images and speed of methods. However, real-time methods are highly unstable resulting in visible flickering when applied to videos. In this work we characterize the instability of these methods by examining the solution set of the style transfer objective. We show that the trace of the Gram matrix representing style is inversely related to the stability of the method. Then, we present a recurrent convolutional network for real-time video style transfer which incorporates a temporal consistency loss and overcomes the instability of prior methods. Our networks can be applied at any resolution, do not re- quire optical flow at test time, and produce high quality, temporally consistent stylized videos in real-time.
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationStyle TransferVideo Style TransferMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Pastiche Master: Exemplar-Based High-Resolution Portrait Style Transfer
Recent studies on StyleGAN show high performance on artistic portrait generation by transfer learning with limited data. In this paper, we explore more challenging exemplar-based high-resolution portrait style transfer b…
Style TransferTransfer LearningVocal Bursts Intensity PredictionGlobal Rhythm Style Transfer Without Text Transcriptions
Prosody plays an important role in characterizing the style of a speaker or an emotion, but most non-parallel voice or emotion style transfer algorithms do not convert any prosody information. Two major components of pro…
Representation LearningRhythmStyle TransferMSSRNet: Manipulating Sequential Style Representation for Unsupervised Text Style Transfer
Unsupervised text style transfer task aims to rewrite a text into target style while preserving its main content. Traditional methods rely on the use of a fixed-sized vector to regulate text style, which is difficult to …
Style TransferText Style TransferUnsupervised Text Style TransferReCoNet: Real-time Coherent Video Style Transfer Network
Image style transfer models based on convolutional neural networks usually suffer from high temporal inconsistency when applied to videos. Some video style transfer models have been proposed to improve temporal consisten…
Semantic SegmentationStyle TransferVideo Style TransferMultiple Style-Transfer in Real-Time
Style transfer aims to combine the content of one image with the artistic style of another. It was discovered that lower levels of convolutional networks captured style information, while higher levels captures content i…
Style Transfer