EfficientQA : a RoBERTa Based Phrase-Indexed Question-Answering System
State-of-the-art extractive question answering models achieve superhuman performances on the SQuAD benchmark. Yet, they are unreasonably heavy and need expensive GPU computing to answer questions in a reasonable time. Thus, they cannot be used for real-world queries on hundreds of thousands of documents in the open-domain question answering paradigm. In this paper, we explore the possibility to transfer the natural language understanding of language models into dense vectors representing questions and answer candidates, in order to make the task of question-answering compatible with a simple nearest neighbor search task. This new model, that we call EfficientQA, takes advantage from the pair of sequences kind of input of BERT-based models to build meaningful dense representations of candidate answers. These latter are extracted from the context in a question-agnostic fashion. Our model achieves state-of-the-art results in Phrase-Indexed Question Answering (PIQA) beating the previous state-of-art by 1.3 points in exact-match and 1.4 points in f1-score. These results show that dense vectors are able to embed very rich semantic representations of sequences, although these ones were built from language models not originally trained for the use-case. Thus, in order to build more resource efficient NLP systems in the future, training language models that are better adapted to build dense representations of phrases is one of the possibilities.
Code (0)
등록된 구현이 없습니다.
Tasks
Extractive Question-AnsweringGPUNatural Language UnderstandingOpen-Domain Question AnsweringQuestion AnsweringSimilar Papers 제목 키워드 기반
Phrase-Indexed Question Answering: A New Challenge for Scalable Document Comprehension
We formalize a new modular variant of current question answering tasks by enforcing complete independence of the document encoder from the question encoder. This formulation addresses a key challenge in machine comprehen…
Question AnsweringReading ComprehensionRetrievalNeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and return natural language answers. The aim…
Open-Domain Question AnsweringQuestion AnsweringRetrievalQuestion Answering Infused Pre-training of General-Purpose Contextualized Representations
We propose a pre-training objective based on question answering (QA) for learning general-purpose contextual representations, motivated by the intuition that the representation of a phrase in a passage should encode all …
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Question Answering+1Question Answering Infused Pre-training of General-Purpose Contextualized Representations
We propose a pre-training objective based on question answering (QA) for learning general-purpose contextual representations, motivated by the intuition that the representation of a phrase in a passage should encode all …
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Question Answering+1Learning Dense Representations of Phrases at Scale
Open-domain question answering can be reformulated as a phrase retrieval problem, without the need for processing documents on-demand during inference (Seo et al., 2019). However, current phrase retrieval models heavily …
Open-Domain Question AnsweringQuestion AnsweringQuestion GenerationReading Comprehension+3