Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis
Due to the difficulty in scaling up, generative adversarial networks (GANs) seem to be falling from grace on the task of text-conditioned image synthesis. Sparsely-activated mixture-of-experts (MoE) has recently been demonstrated as a valid solution to training large-scale models with limited computational resources. Inspired by such a philosophy, we present Aurora, a GAN-based text-to-image generator that employs a collection of experts to learn feature processing, together with a sparse router to help select the most suitable expert for each feature point. To faithfully decode the sampling stochasticity and the text condition to the final synthesis, our router adaptively makes its decision by taking into account the text-integrated global latent code. At 64x64 image resolution, our model trained on LAION2B-en and COYO-700M achieves 6.2 zero-shot FID on MS COCO. We release the code and checkpoints to facilitate the community for further development.
Code (1)
Tasks
Image GenerationMixture-of-ExpertsPhilosophyvalidSimilar Papers 제목 키워드 기반
Creative Portraiture: Exploring Creative Adversarial Networks and Conditional Creative Adversarial Networks
Convolutional neural networks (CNNs) have been combined with generative adversarial networks (GANs) to create deep convolutional generative adversarial networks (DCGANs) with great success. DCGANs have been used for gene…
Hierarchical Modes Exploring in Generative Adversarial Networks
In conditional Generative Adversarial Networks (cGANs), when two different initial noises are concatenated with the same conditional information, the distance between their outputs is relatively smaller, which makes mino…
DiversityImage GenerationText to Image GenerationText-to-Image Generation+1Skip-Thought GAN: Generating Text through Adversarial Training using Skip-Thought Vectors
In the past few years, various advancements have been made in generative models owing to the formulation of Generative Adversarial Networks (GANs). GANs have been shown to perform exceedingly well on a wide variety of ta…
Image GenerationSentenceSentence EmbeddingsStyle Transfer+2Text to Image Synthesis using Stacked Conditional Variational Autoencoders and Conditional Generative Adversarial Networks
Synthesizing a realistic image from textual description is a major challenge in computer vision. Current text to image synthesis approaches falls short of producing a highresolution image that represent a text descriptor…
Image GenerationDistilling GANs with Style-Mixed Triplets for X2I Translation with Limited Data
Conditional image synthesis is an integral part of many X2I translation systems, including image-to-image, text-to-image and audio-to-image translation systems. Training these large systems generally requires huge amount…
Image GenerationKnowledge DistillationSemantic SegmentationTranslation