Leveraging the Wisdom of the Crowds for the Acquisition of Multilingual Language Resources
Games with a purpose are an increasingly popular mechanism for leveraging the wisdom of the crowds to address tasks which are trivial for humans but still not solvable by computer algorithms in a satisfying manner. As a novel mechanism for structuring human-computer interactions, a key challenge when creating them is motivating users to participate while generating useful and unbiased results. This paper focuses on important design choices and success factors of effective games with a purpose. Our findings are based on lessons learned while developing and deploying Sentiment Quiz, a crowdsourcing application for creating sentiment lexicons (an essential component of most sentiment detection algorithms). We describe the goals and structure of the game, the underlying application framework, the sentiment lexicons gathered through crowdsourcing, as well as a novel approach to automatically extend the lexicons by means of a bootstrapping process. Such an automated extension further increases the efficiency of the acquisition process by limiting the number of terms that need to be gathered from the game participants.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Improving Image Clustering using Sparse Text and the Wisdom of the Crowds
We propose a method to improve image clustering using sparse text and the wisdom of the crowds. In particular, we present a method to fuse two different kinds of document features, image and text features, and use a comm…
ClusteringImage ClusteringImproving Spoken Language Understanding by Wisdom of Crowds
Spoken language understanding (SLU), which converts user requests in natural language to machine-interpretable expressions, is becoming an essential task. The lack of training data is an important problem, especially for…
Data AugmentationSpoken Language UnderstandingWisdom from Diversity: Bias Mitigation Through Hybrid Human-LLM Crowds
Despite their performance, large language models (LLMs) can inadvertently perpetuate biases found in the data they are trained on. By analyzing LLM responses to bias-eliciting headlines, we find that these models often m…
DiversityExplainable Fake News Detection With Large Language Model via Defense Among Competing Wisdom
Most fake news detection methods learn latent feature representations based on neural networks, which makes them black boxes to classify a piece of news without giving any justification. Existing explainable systems gene…
Fake News DetectionLanguage ModelingLanguage ModellingLarge Language ModelThe Wisdom of Partisan Crowds: Comparing Collective Intelligence in Humans and LLM-based Agents
Human groups are able to converge on more accurate beliefs through deliberation, even in the presence of polarization and partisan bias -- a phenomenon known as the "wisdom of partisan crowds." Generated agents powered b…