Factor Decomposed Generative Adversarial Networks for Text-to-Image Synthesis
Prior works about text-to-image synthesis typically concatenated the sentence embedding with the noise vector, while the sentence embedding and the noise vector are two different factors, which control the different aspects of the generation. Simply concatenating them will entangle the latent factors and encumber the generative model. In this paper, we attempt to decompose these two factors and propose Factor Decomposed Generative Adversarial Networks~(FDGAN). To achieve this, we firstly generate images from the noise vector and then apply the sentence embedding in the normalization layer for both generator and discriminators. We also design an additive norm layer to align and fuse the text-image features. The experimental results show that decomposing the noise and the sentence embedding can disentangle latent factors in text-to-image synthesis, and make the generative model more efficient. Compared with the baseline, FDGAN can achieve better performance, while fewer parameters are used.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationSentenceSentence EmbeddingSentence-EmbeddingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Style and Content Disentanglement in Generative Adversarial Networks
Disentangling factors of variation within data has become a very challenging problem for image generation tasks. Current frameworks for training a Generative Adversarial Network (GAN), learn to disentangle the representa…
DisentanglementGenerative Adversarial NetworkImage GenerationDeeply Matting-based Dual Generative Adversarial Network for Image and Document Label Supervision
Although many methods have been proposed to deal with nature image super-resolution (SR) and get impressive performance, the text images SR is not good due to their ignorance of document images. In this paper, we propose…
Generative Adversarial NetworkImage MattingImage Super-ResolutionSuper-ResolutionGANDA: A deep generative adversarial network predicts the spatial distribution of nanoparticles in tumor pixelly
Intratumoral nanoparticles (NPs) distribution is critical for the success of nanomedicine in imaging and treatment, but computational models to describe the NPs distribution remain unavailable due to the complex tumor-na…
Generative Adversarial Networkwhole slide imagesDecomposed Adversarial Learned Inference
Effective inference for a generative adversarial model remains an important and challenging problem. We propose a novel approach, Decomposed Adversarial Learned Inference (DALI), which explicitly matches prior and condit…
RetrieverTTS: Modeling Decomposed Factors for Text-Based Speech Insertion
This paper proposes a new "decompose-and-edit" paradigm for the text-based speech insertion task that facilitates arbitrary-length speech insertion and even full sentence generation. In the proposed paradigm, global and …
Sentence