Two-Stream Convolutional Networks for Dynamic Texture Synthesis
We introduce a two-stream model for dynamic texture synthesis. Our model is based on pre-trained convolutional networks (ConvNets) that target two independent tasks: (i) object recognition, and (ii) optical flow prediction. Given an input dynamic texture, statistics of filter responses from the object recognition ConvNet encapsulate the per-frame appearance of the input texture, while statistics of filter responses from the optical flow ConvNet model its dynamics. To generate a novel texture, a randomly initialized input sequence is optimized to match the feature statistics from each stream of an example texture. Inspired by recent work on image style transfer and enabled by the two-stream model, we also apply the synthesis approach to combine the texture appearance from one texture with the dynamics of another to generate entirely novel dynamic textures. We show that our approach generates novel, high quality samples that match both the framewise appearance and temporal evolution of input texture. Finally, we quantitatively evaluate our texture synthesis approach with a thorough user study.
Code (1)
Tasks
Object RecognitionOptical Flow EstimationStyle TransferTexture SynthesisVocal Bursts Valence PredictionSimilar Papers 제목 키워드 기반
Synthesising Dynamic Textures using Convolutional Neural Networks
Here we present a parametric model for dynamic textures. The model is based on spatiotemporal summary statistics computed from the feature representations of a Convolutional Neural Network (CNN) trained on object recogni…
Object RecognitionNeural FFTs for Universal Texture Image Synthesis
Synthesizing larger texture images from a smaller exemplar is an important task in graphics and vision. The conventional CNNs, recently adopted for synthesis, require to train and test on the same set of images and fail …
Image GenerationTexture SynthesisTexture Synthesis Using Shallow Convolutional Networks with Random Filters
Here we demonstrate that the feature space of random shallow convolutional neural networks (CNNs) can serve as a surprisingly good model of natural textures. Patches from the same texture are consistently classified as b…
Texture SynthesisDynamic Texture Synthesis by Incorporating Long-range Spatial and Temporal Correlations
The main challenge of dynamic texture synthesis lies in how to maintain spatial and temporal consistency in synthesized videos. The major drawback of existing dynamic texture synthesis models comes from poor treatment of…
Texture SynthesisConditional Generative ConvNets for Exemplar-based Texture Synthesis
The goal of exemplar-based texture synthesis is to generate texture images that are visually similar to a given exemplar. Recently, promising results have been reported by methods relying on convolutional neural networks…
Texture Synthesis