Collecting Image Description Datasets using Crowdsourcing
We describe our two new datasets with images described by humans. Both the datasets were collected using Amazon Mechanical Turk, a crowdsourcing platform. The two datasets contain significantly more descriptions per image than other existing datasets. One is based on a popular image description dataset called the UIUC Pascal Sentence Dataset, whereas the other is based on the Abstract Scenes dataset con- taining images made from clipart objects. In this paper we describe our interfaces, analyze some properties of and show example descriptions from our two datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Image DescriptionSentenceSimilar Papers 제목 키워드 기반
Face2Text: Collecting an Annotated Image Description Corpus for the Generation of Rich Face Descriptions
The past few years have witnessed renewed interest in NLP tasks at the interface between vision and language. One intensively-studied problem is that of automatically generating text from images. In this paper, we extend…
Image DescriptionImage to textPredicting Foreground Object Ambiguity and Efficiently Crowdsourcing the Segmentation(s)
We propose the ambiguity problem for the foreground object segmentation task and motivate the importance of estimating and accounting for this ambiguity when designing vision systems. Specifically, we distinguish between…
DiversityObjectSemantic SegmentationvalidWhat Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks?
Crowdsourcing is widely used to create data for common natural language understanding tasks. Despite the importance of these datasets for measuring and refining model understanding of language, there has been little focu…
Multiple-choiceNatural Language UnderstandingQuestion AnsweringA Comparative Study on Annotation Quality of Crowdsourcing and LLM via Label Aggregation
Whether Large Language Models (LLMs) can outperform crowdsourcing on the data annotation task is attracting interest recently. Some works verified this issue with the average performance of individual crowd workers and L…
Ballpark Crowdsourcing: The Wisdom of Rough Group Comparisons
Crowdsourcing has become a popular method for collecting labeled training data. However, in many practical scenarios traditional labeling can be difficult for crowdworkers (for example, if the data is high-dimensional or…
BIG-bench Machine Learning