MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
Retrieval question answering (ReQA) is the task of retrieving a sentence-level answer to a question from an open corpus (Ahmad et al.,2019).This paper presents MultiReQA, anew multi-domain ReQA evaluation suite com-posed of eight retrieval QA tasks drawn from publicly available QA datasets. We provide the first systematic retrieval based evaluation over these datasets using two supervised neural models, based on fine-tuning BERT andUSE-QA models respectively, as well as a surprisingly strong information retrieval baseline,BM25. Five of these tasks contain both train-ing and test data, while three contain test data only. Performance on the five tasks with train-ing data shows that while a general model covering all domains is achievable, the best performance is often obtained by training exclusively on in-domain data.
Code (1)
Tasks
Information RetrievalQuestion AnsweringRetrievalSentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MultiReQA: A Cross-Domain Evaluation forRetrieval Question Answering Models
Retrieval question answering (ReQA) is the task of retrieving a sentence-level answer to a question from an open corpus (Ahmad et al.,2019).This dataset paper presents MultiReQA, a new multi-domain ReQA evaluation suite …
Information RetrievalQuestion AnsweringRetrievalSentence+1Exploring Dual Encoder Architectures for Question Answering
Dual encoders have been used for question-answering (QA) and information retrieval (IR) tasks with good results. Previous research focuses on two major types of dual encoders, Siamese Dual Encoder (SDE), with parameters …
Information RetrievalQuestion AnsweringRetrievalSamToNe: Improving Contrastive Loss for Dual Encoder Retrieval Models with Same Tower Negatives
Dual encoders have been used for retrieval tasks and representation learning with good results. A standard way to train dual encoders is using a contrastive loss with in-batch negatives. In this work, we propose an impro…
Contrastive LearningInformation RetrievalQuestion AnsweringRepresentation Learning+1Retrieval-Augmented Generation for Reliable Interpretation of Radio Regulations
We study question answering in the domain of radio regulations, a legally sensitive and high-stakes area. We propose a telecom-specific Retrieval-Augmented Generation (RAG) pipeline and introduce, to our knowledge, the f…
Question AnsweringAGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs
Generating diverse and effective clarifying questions is crucial for improving query understanding and retrieval performance in open-domain conversational search (CS) systems. We propose AGENT-CQ (Automatic GENeration, a…
Conversational SearchRetrieval