Can adversarial training learn image captioning ?
Recently, generative adversarial networks (GAN) have gathered a lot of interest. Their efficiency in generating unseen samples of high quality, especially images, has improved over the years. In the field of Natural Language Generation (NLG), the use of the adversarial setting to generate meaningful sentences has shown to be difficult for two reasons: the lack of existing architectures to produce realistic sentences and the lack of evaluation tools. In this paper, we propose an adversarial architecture related to the conditional GAN (cGAN) that generates sentences according to a given image (also called image captioning). This attempt is the first that uses no pre-training or reinforcement methods. We also explain why our experiment settings can be safely evaluated and interpreted for further works.
Code (1)
Tasks
Image CaptioningText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning Compact Reward for Image Captioning
Adversarial learning has shown its advances in generating natural and diverse descriptions in image captioning. However, the learned reward of existing adversarial methods is vague and ill-defined due to the reward ambig…
DiversityImage CaptioningReinforcement LearningReinforcement Learning (RL)+1Exact Adversarial Attack to Image Captioning via Structured Output Learning with Latent Variables
In this work, we study the robustness of a CNN+RNN based image captioning system being subjected to adversarial noises. We propose to fool an image captioning system to generate some targeted partial captions for an imag…
Adversarial AttackImage CaptioningAttacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning
Visual language grounding is widely studied in modern neural image captioning systems, which typically adopts an encoder-decoder framework consisting of two principal components: a convolutional neural network (CNN) for …
Caption GenerationDecoderImage CaptioningImproving Image Captioning with Conditional Generative Adversarial Nets
In this paper, we propose a novel conditional-generative-adversarial-nets-based image captioning framework as an extension of traditional reinforcement-learning (RL)-based encoder-decoder architecture. To deal with the i…
DecoderImage CaptioningReinforcement LearningReinforcement Learning (RL)Improved Adversarial Image Captioning
In this paper we study image captioning as a conditional GAN training, proposing both a context-aware LSTM captioner and co-attentive discriminator, which enforces semantic alignment between images and captions. We inves…
Image Captioning