A Collection of Question Answering Datasets for Norwegian
This paper introduces a new suite of question answering datasets for Norwegian; NorOpenBookQA, NorCommonSenseQA, NorTruthfulQA, and NRK-Quiz-QA. The data covers a wide range of skills and knowledge domains, including world knowledge, commonsense reasoning, truthfulness, and knowledge about Norway. Covering both of the written standards of Norwegian - Bokm{\aa}l and Nynorsk - our datasets comprise over 10k question-answer pairs, created by native speakers. We detail our dataset creation approach and present the results of evaluating 11 language models (LMs) in zero- and few-shot regimes. Most LMs perform better in Bokm{\aa}l than Nynorsk, struggle most with commonsense reasoning, and are often untruthful in generating answers to questions. All our datasets and annotation materials are publicly available.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringWorld KnowledgeSimilar Papers 제목 키워드 기반
NorQuAD: Norwegian Question Answering Dataset
In this paper we present NorQuAD: the first Norwegian question answering dataset for machine reading comprehension. The dataset consists of 4,752 manually created question-answer pairs. We here detail the data collection…
Machine Reading ComprehensionQuestion AnsweringReading ComprehensionNorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark
This paper introduces NorEval, a new and comprehensive evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs). NorEval consists of 24 high-quality human-created datasets …
BenchmarkingBRENT: Bidirectional Retrieval Enhanced Norwegian Transformer
Retrieval-based language models are increasingly employed in question-answering tasks. These models search in a corpus of documents for relevant information instead of having all factual knowledge stored in its parameter…
Dependency ParsingExtractive Question-AnsweringLanguage ModelingLanguage Modelling+6ArchivalQA: A Large-scale Benchmark Dataset for Open Domain Question Answering over Archival News Collections
In the last few years, open-domain question answering (ODQA) has advanced rapidly due to the development of deep learning techniques and the availability of large-scale QA datasets. However, the current datasets are esse…
Open-Domain Question AnsweringQuestion AnsweringChroniclingAmericaQA: A Large-scale Question Answering Dataset based on Historical American Newspaper Pages
Question answering (QA) and Machine Reading Comprehension (MRC) tasks have significantly advanced in recent years due to the rapid development of deep learning techniques and, more recently, large language models. At the…
Machine Reading ComprehensionOptical Character Recognition (OCR)Question AnsweringReading Comprehension