Alleviating Noisy Data in Image Captioning with Cooperative Distillation
Image captioning systems have made substantial progress, largely due to the availability of curated datasets like Microsoft COCO or Vizwiz that have accurate descriptions of their corresponding images. Unfortunately, scarce availability of such cleanly labeled data results in trained algorithms producing captions that can be terse and idiosyncratically specific to details in the image. We propose a new technique, cooperative distillation that combines clean curated datasets with the web-scale automatically extracted captions of the Google Conceptual Captions dataset (GCC), which can have poor descriptions of images, but is abundant in size and therefore provides a rich vocabulary resulting in more expressive captions.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningSimilar Papers 제목 키워드 기반
Non-Autoregressive Image Captioning with Counterfactuals-Critical Multi-Agent Learning
Most image captioning models are autoregressive, i.e. they generate each word by conditioning on previously generated words, which leads to heavy latency during inference. Recently, non-autoregressive decoding has been p…
Image CaptioningMachine TranslationMulti-agent Reinforcement LearningSentence+1Perturb, Predict & Paraphrase: Semi-Supervised Learning using Noisy Student for Image Captioning
Recent semi-supervised learning (SSL) methods are predominantly focused on multi-class classification tasks. Classification tasks allow for easy mixing of class labels during augmentation which does not trivially extend …
Data AugmentationImage AugmentationImage Captioningimage-classification+1Emergent Natural Language with Communication Games for Improving Image Captioning Capabilities without Additional Data
Image captioning is an important problem in developing various AI systems, and these tasks require large volumes of annotated images to train the models. Since all existing labelled datasets are already used for training…
Multi-agent Reinforcement LearningImage CaptioningEnd-to-end Image Captioning Exploits Multimodal Distributional Similarity
We hypothesize that end-to-end neural image captioning systems work seemingly well because they exploit and learn `distributional similarity' in a multimodal feature space by mapping a test image to similar training imag…
Image CaptioningText GenerationLMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting
Multilingual image captioning has recently been tackled by training with large-scale machine translated data, which is an expensive, noisy, and time-consuming process. Without requiring any multilingual caption data, we …
DecoderImage CaptioningLanguage ModelingLanguage Modelling+1