paper-with-me

홈 › Papers

Autocompletion interfaces make crowd workers slower, but their use promotes response diversity

2017-07-21 · Xipei Liu, James P. Bagrow

Creative tasks such as ideation or question proposal are powerful applications of crowdsourcing, yet the quantity of workers available for addressing practical problems is often insufficient. To enable scalable crowdsourcing thus requires gaining all possible efficiency and information from available workers. One option for text-focused tasks is to allow assistive technology, such as an autocompletion user interface (AUI), to help workers input text responses. But support for the efficacy of AUIs is mixed. Here we designed and conducted a randomized experiment where workers were asked to provide short text responses to given questions. Our experimental goal was to determine if an AUI helps workers respond more quickly and with improved consistency by mitigating typos and misspellings. Surprisingly, we found that neither occurred: workers assigned to the AUI treatment were slower than those assigned to the non-AUI control and their responses were more diverse, not less, than those of the control. Both the lexical and semantic diversities of responses were higher, with the latter measured using word2vec. A crowdsourcer interested in worker speed may want to avoid using an AUI, but using an AUI to boost response diversity may be valuable to crowdsourcers interested in receiving as much novel information from workers as possible.

📄 PDF Abstract BibTeX arXiv:1707.06939

Code (0)

등록된 구현이 없습니다.

Tasks

Diversity

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Similar Papers 제목 키워드 기반

If in a Crowdsourced Data Annotation Pipeline, a GPT-4

2024-02-26 · Zeyu He, Chieh-Yang Huang, Chien-Kuang Cornelia Ding, Shaurya Rohatgi 외

Recent studies indicated GPT-4 outperforms online crowd workers in data labeling accuracy, notably workers from Amazon Mechanical Turk (MTurk). However, these studies were criticized for deviating from standard crowdsour…

ArticlesSentence

GWLAN: General Word-Level AutocompletioN for Computer-Aided Translation

2021-05-31 · ACL 2021 5 · Huayang Li, Lemao Liu, Guoping Huang, Shuming Shi

Computer-aided translation (CAT), the use of software to assist a human translator in the translation process, has been proven to be useful in enhancing the productivity of human translators. Autocompletion, which sugges…

SentenceTranslation

From Zero to The Hero: A Collaborative Market Aware Recommendation System for Crowd Workers

2021-07-06 · Hamid Shamszare, Razieh Saremi, Sanam Jena

The success of software crowdsourcing depends on active and trustworthy pool of worker supply. The uncertainty of crowd workers' behaviors makes it challenging to predict workers' success and plan accordingly. In a compe…

Decision Making

Hear Me Out: A Study on the Use of the Voice Modality for Crowdsourced Relevance Assessments

2023-04-21 · Nirmal Roy, Agathe Balayn, David Maxwell, Claudia Hauff

The creation of relevance assessments by human assessors (often nowadays crowdworkers) is a vital step when building IR test collections. Prior works have investigated assessor quality & behaviour, though into the impact…

Time-Sensitive Bayesian Information Aggregation for Crowdsourcing Systems

2015-10-21 · Matteo Venanzi, John Guiver, Pushmeet Kohli, Nick Jennings

Crowdsourcing systems commonly face the problem of aggregating multiple judgments provided by potentially unreliable workers. In addition, several aspects of the design of efficient crowdsourcing processes, such as defin…

Bayesian InferenceEntity Linkingvalid