Harvesting Paragraph-Level Question-Answer Pairs from Wikipedia
We study the task of generating from Wikipedia articles question-answer pairs that cover content beyond a single sentence. We propose a neural network approach that incorporates coreference knowledge via a novel gating mechanism. Compared to models that only take into account sentence-level information (Heilman and Smith, 2010; Du et al., 2017; Zhou et al., 2017), we find that the linguistic knowledge introduced by the coreference representation aids question generation significantly, producing models that outperform the current state-of-the-art. We apply our system (composed of an answer span extraction system and the passage-level QG system) to the 10,000 top-ranking Wikipedia articles and create a corpus of over one million question-answer pairs. We also provide a qualitative analysis for this large-scale generated corpus from Wikipedia.
Code (1)
Tasks
ArticlesQuestion GenerationQuestion-GenerationSentenceSimilar Papers 제목 키워드 기반
On the Evaluation of Answer-Agnostic Paragraph-level Multi-Question Generation
We study the task of predicting a set of salient questions from a given paragraph without any prior knowledge of the precise answer. We make two main contributions. First, we propose a new method to evaluate a set of pre…
Question GenerationQuestion-GenerationMultilingual Non-Factoid Question Answering with Answer Paragraph Selection
Most existing Question Answering Datasets (QuADs) primarily focus on factoid-based short-context Question Answering (QA) in high-resource languages. However, the scope of such datasets for low-resource languages remains …
ArticlesQuestion AnsweringLegal Question Answering using Ranking SVM and Deep Convolutional Neural Network
This paper presents a study of employing Ranking SVM and Convolutional Neural Network for two missions: legal information retrieval and question answering in the Competition on Legal Information Extraction/Entailment. Fo…
Information RetrievalQuestion AnsweringRetrievalWikiOmnia: generative QA corpus on the whole Russian Wikipedia
The General QA field has been developing the methodology referencing the Stanford Question answering dataset (SQuAD) as the significant benchmark. However, compiling factual questions is accompanied by time- and labour-c…
Question AnsweringMulti-Hop Paragraph Retrieval for Open-Domain Question Answering
This paper is concerned with the task of multi-hop open-domain Question Answering (QA). This task is particularly challenging since it requires the simultaneous performance of textual reasoning and efficient searching. W…
Open-Domain Question AnsweringQuestion AnsweringRetrievalSentence