Crowdsourcing Question-Answer Meaning Representations
We introduce Question-Answer Meaning Representations (QAMRs), which represent the predicate-argument structure of a sentence as a set of question-answer pairs. We also develop a crowdsourcing scheme to show that QAMRs can be labeled with very little training, and gather a dataset with over 5,000 sentences and 100,000 questions. A detailed qualitative analysis demonstrates that the crowd-generated question-answer pairs cover the vast majority of predicate-argument relationships in existing datasets (including PropBank, NomBank, QA-SRL, and AMR) along with many previously under-resourced ones, including implicit arguments and relations. The QAMR data and annotation code is made publicly available to enable future work on how best to model these complex phenomena.
Code (1)
Tasks
SentenceSimilar Papers 제목 키워드 기반
SelQA: A New Benchmark for Selection-based Question Answering
This paper presents a new selection-based question answering dataset, SelQA. The dataset consists of questions generated through crowdsourcing and sentence length answers that are drawn from the ten most prevalent topics…
Question AnsweringSentenceBreak It Down: A Question Understanding Benchmark
Understanding natural language questions entails the ability to break down a question into the requisite steps for computing its answer. In this work, we introduce a Question Decomposition Meaning Representation (QDMR) f…
Open-Domain Question AnsweringQuestion AnsweringReading ComprehensionSemantic Parsing+1Distinguishing Question Subjectivity from Difficulty for Improved Crowdsourcing
The questions in a crowdsourcing task typically exhibit varying degrees of difficulty and subjectivity. Their joint effects give rise to the variation in responses to the same question by different crowd-workers. This va…
ClusteringsloWCrowd: A crowdsourcing tool for lexicographic tasks
The paper presents sloWCrowd, a simple tool developed to facilitate crowdsourcing lexicographic tasks, such as error correction in automatically generated wordnets and semantic annotation of corpora. The tool is open-sou…
Visual Question: Predicting If a Crowd Will Agree on the Answer
Visual question answering (VQA) systems are emerging from a desire to empower users to ask any natural language question about visual content and receive a valid answer in response. However, close examination of the VQA …
Question AnsweringvalidVisual Question AnsweringVisual Question Answering (VQA)