Adversarial Learning of Semantic Relevance in Text to Image Synthesis
We describe a new approach that improves the training of generative adversarial nets (GANs) for synthesizing diverse images from a text input. Our approach is based on the conditional version of GANs and expands on previous work leveraging an auxiliary task in the discriminator. Our generated images are not limited to certain classes and do not suffer from mode collapse while semantically matching the text input. A key to our training methods is how to form positive and negative training examples with respect to the class label of a given image. Instead of selecting random training examples, we perform negative sampling based on the semantic distance from a positive example in the class. We evaluate our approach using the Oxford-102 flower dataset, adopting the inception score and multi-scale structural similarity index (MS-SSIM) metrics to assess discriminability and diversity of the generated images. The empirical results indicate greater diversity in the generated images, especially when we gradually select more negative training examples closer to a positive example in the semantic space.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityImage GenerationMS-SSIMSSIMSimilar Papers 제목 키워드 기반
Image Synthesis with Class-Aware Semantic Diffusion Models for Surgical Scene Segmentation
Surgical scene segmentation is essential for enhancing surgical precision, yet it is frequently compromised by the scarcity and imbalance of available data. To address these challenges, semantic image synthesis methods b…
Image GenerationScene SegmentationSegmentationYou Only Need Adversarial Supervision for Semantic Image Synthesis
Despite their recent successes, GAN models for semantic image synthesis still suffer from poor image quality when trained with only adversarial supervision. Historically, additionally employing the VGG-based perceptual l…
Image GenerationImage-to-Image TranslationSemantic Segmentationsemantic image synthesis of anime characters based on conditional generative adversarial networks
The goal of semantic image synthesis is to generate realistic images from semantic label maps. However, current approaches for generating anime characters from semantic label maps still encounter some issues, particularl…
Edge DetectionGenerative Adversarial NetworkImage GenerationSemantic SegmentationSAM-GAN: Self-Attention supporting Multi-stage Generative Adversarial Networks for text-to-image synthesis
Synthesizing photo-realistic images based on text descriptions is a challenging task in the field of computer vision. Although generative adversarial networks have made significant breakthroughs in this task, they stil…
Image GenerationSemantic SimilaritySemantic Textual SimilaritySentenceAdversarial Text-to-Image Synthesis: A Review
With the advent of generative adversarial networks, synthesizing images from textual descriptions has recently become an active research area. It is a flexible and intuitive way for conditional image generation with sign…
Adversarial TextConditional Image GenerationDiversityImage Generation