ELQA: A Corpus of Metalinguistic Questions and Answers about English
We present ELQA, a corpus of questions and answers in and about the English language. Collected from two online forums, the >70k questions (from English learners and others) cover wide-ranging topics including grammar, meaning, fluency, and etymology. The answers include descriptions of general properties of English vocabulary and grammar as well as explanations about specific (correct and incorrect) usage examples. Unlike most NLP datasets, this corpus is metalinguistic -- it consists of language about language. As such, it can facilitate investigations of the metalinguistic capabilities of NLU models, as well as educational applications in the language learning domain. To study this, we define a free-form question answering task on our dataset and conduct evaluations on multiple LLMs (Large Language Models) to analyze their capacity to generate metalinguistic answers.
Code (1)
Tasks
Answer GenerationQuestion AnsweringSimilar Papers 제목 키워드 기반
SelQA: A New Benchmark for Selection-based Question Answering
This paper presents a new selection-based question answering dataset, SelQA. The dataset consists of questions generated through crowdsourcing and sentence length answers that are drawn from the ten most prevalent topics…
Question AnsweringSentenceEvaluating Metalinguistic Knowledge in Large Language Models across the World's Languages
LLMs are routinely evaluated on language use, yet their explicit knowledge about linguistic structure remains poorly understood. Existing linguistic benchmarks focus on narrow phenomena, emphasize high-resource languages…
Towards Inference-Oriented Reading Comprehension: ParallelQA
In this paper, we investigate the tendency of end-to-end neural Machine Reading Comprehension (MRC) models to match shallow patterns rather than perform inference-oriented reasoning on RC benchmarks. We aim to test the a…
Machine Reading ComprehensionReading ComprehensionNovelQA: Benchmarking Question Answering on Documents Exceeding 200K Tokens
The rapid advancement of Large Language Models (LLMs) has introduced a new frontier in natural language processing, particularly in understanding and processing long-context information. However, the evaluation of these …
BenchmarkingQuestion AnsweringCausalQA: A Benchmark for Causal Question Answering
At least 5% of questions submitted to search engines ask about cause-effect relationships in some way. To support the development of tailored approaches that can answer such questions, we construct Webis-CausalQA-22, a b…
Question Answering