Joint Models for Answer Verification in Question Answering Systems
This paper studies joint models for selecting correct answer sentences among the top $k$ provided by answer sentence selection (AS2) modules, which are core components of retrieval-based Question Answering (QA) systems. Our work shows that a critical step to effectively exploit an answer set regards modeling the interrelated information between pair of answers. For this purpose, we build a three-way multi-classifier, which decides if an answer supports, refutes, or is neutral with respect to another one. More specifically, our neural architecture integrates a state-of-the-art AS2 model with the multi-classifier, and a joint layer connecting all components. We tested our models on WikiQA, TREC-QA, and a real-world dataset. The results show that our models obtain the new state of the art in AS2.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringRetrievalSentenceSimilar Papers 제목 키워드 기반
Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering
Many Question-Answering (QA) datasets contain unanswerable questions, but their treatment in QA systems remains primitive. Our analysis of the Natural Questions (Kwiatkowski et al. 2019) dataset reveals that a substantia…
Explanation GenerationNatural QuestionsQuestion AnsweringBenchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics
Question answering-based summarization evaluation metrics must automatically determine whether the QA model's prediction is correct or not, a task known as answer verification. In this work, we benchmark the lexical answ…
AttributeBenchmarkingQuestion AnsweringBenchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics
Question answering-based summarization evaluation metrics must automatically determine whether the QA model's prediction is correct or not, a task known as answer verification. In this work, we benchmark the lexical answ…
AttributeBenchmarkingQuestion AnsweringDouble Retrieval and Ranking for Accurate Question Answering
Recent work has shown that an answer verification step introduced in Transformer-based answer selection models can significantly improve the state of the art in Question Answering. This step is performed by aggregating t…
Answer SelectionQuestion AnsweringRerankingRetrievalCalibration of Machine Reading Systems at Scale
In typical machine learning systems, an estimate of the probability of the prediction is used to assess the system's confidence in the prediction. This confidence measure is usually uncalibrated; i.e.\ the system's confi…
Claim VerificationOpen-Domain Question AnsweringPredictionQuestion Answering+2