Visual Madlibs: Fill in the blank Image Generation and Question Answering
In this paper, we introduce a new dataset consisting of 360,001 focused natural language descriptions for 10,738 images. This dataset, the Visual Madlibs dataset, is collected using automatically produced fill-in-the-blank templates designed to gather targeted descriptions about: people and objects, their appearances, activities, and interactions, as well as inferences about the general scene or its broader context. We provide several analyses of the Visual Madlibs dataset and demonstrate its applicability to two new description generation tasks: focused description generation, and multiple-choice question-answering for images. Experiments using joint-embedding and deep learning methods show promising results on these tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationMultiple-choiceQuestion AnsweringSimilar Papers 제목 키워드 기반
Visual Madlibs: Fill in the Blank Description Generation and Question Answering
In this paper, we introduce a new dataset consisting of 360,001 focused natural language descriptions for 10,738 images. This dataset, the Visual Madlibs dataset, is collected using automatically produced fill-in-the-bl…
Multiple-choiceQuestion AnsweringSolving Visual Madlibs with Multiple Cues
This paper focuses on answering fill-in-the-blank style multiple choice questions from the Visual Madlibs dataset. Previous approaches to Visual Question Answering (VQA) have mainly used generic image features from netwo…
Activity PredictionAttributeMultiple-choiceQuestion Answering+3Combining Multiple Cues for Visual Madlibs Question Answering
This paper presents an approach for answering fill-in-the-blank multiple choice questions from the Visual Madlibs dataset. Instead of generic and commonly used representations trained on the ImageNet classification task,…
AttributeGeneral ClassificationMultiple-choiceQuestion Answering+1Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning
We develop the first approximate inference algorithm for 1-Best (and M-Best) decoding in bidirectional neural sequence models by extending Beam Search (BS) to reason about both forward and backward time dependencies. Bea…
Image CaptioningSentenceLearning Models for Actions and Person-Object Interactions with Transfer to Question Answering
This paper proposes deep convolutional network models that utilize local and global context to make human activity label predictions in still images, achieving state-of-the-art performance on two recent datasets with hun…
General ClassificationHuman-Object Interaction DetectionMultiple-choiceMultiple Instance Learning+3