AmbigNQ

홈페이지 · 논문 11편

The AmbigNQ dataset is a valuable resource for exploring ambiguity in open-domain question answering. Let me provide you with some details: 1. Task Description: - Ambiguity is inherent in open-domain question answering, especially when dealing with new topics. It can be challenging to formulate questions that have a single, unambiguous answer. - The AmbigQA task involves predicting a set of question-answer pairs, where each plausible answer is paired with a disambiguated rewrite of the original question. 2. Dataset Construction: - To study this task, the researchers constructed the AmbigNQ dataset. - AmbigNQ covers 14,042 questions from NQ-open, which is an existing open-domain QA benchmark. - Surprisingly, over half of the questions in NQ-open exhibit ambiguity. - The types of ambiguity are diverse and sometimes subtle, often becoming apparent only after examining evidence provided by a very large text corpus. 3. Dataset Versions: - There are three versions of the AmbigNQ dataset: - Light Version: Contains only inputs and outputs. - Full Version: Includes all annotation metadata. - Evidence Version: Provides semi-oracle evidence articles along with questions and answers. (1) AmbigQA - University of Washington. https://nlp.cs.washington.edu/ambigqa/. (2) ambig_qa.py · ambig_qa at main - Hugging Face. https://huggingface.co/datasets/ambig_qa/blob/main/ambig_qa.py. (3) dataset_infos.json · ambig_qa at main - Hugging Face. https://huggingface.co/datasets/ambig_qa/blob/main/dataset_infos.json. (4) AmbigQA/AmbigNQ README - GitHub: Let’s build from here. https://github.com/shmsw25/AmbigQA.