Improved Adversarial Image Captioning
In this paper we study image captioning as a conditional GAN training, proposing both a context-aware LSTM captioner and co-attentive discriminator, which enforces semantic alignment between images and captions. We investigate the viability of two discrete GAN training methods: Self-critical Sequence Training (SCST) and Gumbel Straight-Through (ST) and demonstrate that SCST shows more stable gradient behavior and improved results over Gumbel ST.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Can adversarial training learn image captioning ?
Recently, generative adversarial networks (GAN) have gathered a lot of interest. Their efficiency in generating unseen samples of high quality, especially images, has improved over the years. In the field of Natural Lang…
Image CaptioningText GenerationAI Safety in Practice: Enhancing Adversarial Robustness in Multimodal Image Captioning
Multimodal machine learning models that combine visual and textual data are increasingly being deployed in critical applications, raising significant safety and security concerns due to their vulnerability to adversarial…
Adversarial RobustnessComputational EfficiencyDecoderImage CaptioningExact Adversarial Attack to Image Captioning via Structured Output Learning with Latent Variables
In this work, we study the robustness of a CNN+RNN based image captioning system being subjected to adversarial noises. We propose to fool an image captioning system to generate some targeted partial captions for an imag…
Adversarial AttackImage CaptioningAttacking Visual Language Grounding with Adversarial Examples: A Case Study on Neural Image Captioning
Visual language grounding is widely studied in modern neural image captioning systems, which typically adopts an encoder-decoder framework consisting of two principal components: a convolutional neural network (CNN) for …
Caption GenerationDecoderImage CaptioningAdversarial Semantic Alignment for Improved Image Captions
In this paper we study image captioning as a conditional GAN training, proposing both a context-aware LSTM captioner and co-attentive discriminator, which enforces semantic alignment between images and captions. We empir…
Image Captioning