Style Generation: Image Synthesis based on Coarsely Matched Texts
Previous text-to-image synthesis algorithms typically use explicit textual instructions to generate/manipulate images accurately, but they have difficulty adapting to guidance in the form of coarsely matched texts. In this work, we attempt to stylize an input image using such coarsely matched text as guidance. To tackle this new problem, we introduce a novel task called text-based style generation and propose a two-stage generative adversarial network: the first stage generates the overall image style with a sentence feature, and the second stage refines the generated style with a synthetic feature, which is produced by a multi-modality style synthesis module. We re-filter one existing dataset and collect a new dataset for the task. Extensive experiments and ablation studies are conducted to validate our framework. The practical potential of our work is demonstrated by various applications such as text-image alignment and story visualization. Our datasets are published at https://www.kaggle.com/datasets/mengyaocui/style-generation.
Code (0)
등록된 구현이 없습니다.
Tasks
Generative Adversarial NetworkImage GenerationSentenceStory VisualizationSimilar Papers 제목 키워드 기반
A spatiotemporal style transfer algorithm for dynamic visual stimulus generation
Understanding how visual information is encoded in biological and artificial systems often requires vision scientists to generate appropriate stimuli to test specific hypotheses. Although deep neural network models have …
Image GenerationObject RecognitionStyle TransferVideo GenerationMPG: A Multi-ingredient Pizza Image Generator with Conditional StyleGANs
Multilabel conditional image generation is a challenging problem in computer vision. In this work we propose Multi-ingredient Pizza Generator (MPG), a conditional Generative Neural Network (GAN) framework for synthesizin…
Conditional Image GenerationImage GenerationCSGO: Content-Style Composition in Text-to-Image Generation
The diffusion model has shown exceptional capabilities in controlled image generation, which has further fueled interest in image style transfer. Existing works mainly focus on training free-based methods (e.g., image in…
Image GenerationStyle TransferText to Image GenerationText-to-Image GenerationUnsupervised Pose Flow Learning for Pose Guided Synthesis
Pose guided synthesis aims to generate a new image in an arbitrary target pose while preserving the appearance details from the source image. Existing approaches rely on either hard-coded spatial transformations or 3D bo…
Using multiple reference audios and style embedding constraints for speech synthesis
The end-to-end speech synthesis model can directly take an utterance as reference audio, and generate speech from the text with prosody and speaker characteristics similar to the reference audio. However, an appropriate …
SentenceSentence SimilaritySpeech Synthesis