CPGAN: Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis
Typical methods for text-to-image synthesis seek to design effective generative architecture to model the text-to-image mapping directly. It is fairly arduous due to the cross-modality translation. In this paper we circumvent this problem by focusing on parsing the content of both the input text and the synthesized image thoroughly to model the text-to-image consistency in the semantic level. Particularly, we design a memory structure to parse the textual content by exploring semantic correspondence between each word in the vocabulary to its various visual contexts across relevant images during text encoding. Meanwhile, the synthesized image is parsed to learn its semantics in an object-aware manner. Moreover, we customize a conditional discriminator to model the fine-grained correlations between words and image sub-regions to push for the text-image semantic alignment. Extensive experiments on COCO dataset manifest that our model advances the state-of-the-art performance significantly (from 35.69 to 52.73 in Inception Score).
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationSemantic correspondenceTranslationSimilar Papers 제목 키워드 기반
CPGAN: Full-Spectrum Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis
Typical methods for text-to-image synthesis seek to design effective generative architecture to model the text-to-image mapping directly. It is fairly arduous due to the cross-modality translation. In this paper we circu…
Image GenerationSemantic correspondenceTranslationBrain Stroke Lesion Segmentation Using Consistent Perception Generative Adversarial Network
The state-of-the-art deep learning methods have demonstrated impressive performance in segmentation tasks. However, the success of these methods depends on a large amount of manually labeled masks, which are expensive an…
Generative Adversarial NetworkLesion SegmentationSegmentationProfile to Frontal Face Recognition in the Wild Using Coupled Conditional GAN
In recent years, with the advent of deep-learning, face recognition has achieved exceptional success. However, many of these deep face recognition models perform much better in handling frontal faces compared to profile …
Domain AdaptationFace RecognitionGenerative Adversarial NetworkRetrievalSimulation of collision avoidance behavior in crowd movement by data-driven approach
Crowd movement simulation is essential for pedestrian safety management and facility layout optimization. Data-driven models enhance trajectory prediction accuracy under Euclidean metrics, yet they suffer from excessivel…
Trajectory PredictionCollision AvoidancePF-cpGAN: Profile to Frontal Coupled GAN for Face Recognition in the Wild
In recent years, due to the emergence of deep learning, face recognition has achieved exceptional success. However, many of these deep face recognition models perform relatively poorly in handling profile faces compared …
Face RecognitionGenerative Adversarial NetworkRetrieval