RECONSIDER: Improved Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negative examples. This training scheme possibly explains empirical observations that these models achieve a high recall amongst their top few predictions, but a low overall accuracy, motivating the need for answer re-ranking. We develop a successful re-ranking approach (RECONSIDER) for span-extraction tasks that improves upon the performance of MRC models, even beyond large-scale pre-training. RECONSIDER is trained on positive and negative examples extracted from high confidence MRC model predictions, and uses in-passage span annotations to perform span-focused re-ranking over a smaller candidate set. As a result, RECONSIDER learns to eliminate close false positives, achieving a new extractive state of the art on four QA tasks, with 45.5{\%} Exact Match accuracy on Natural Questions with real user questions, and 61.7{\%} on TriviaQA. We will release all related data, models, and code.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine Reading ComprehensionNatural QuestionsOpen-Domain Question AnsweringQuestion AnsweringReading ComprehensionRe-RankingTriviaQASimilar Papers 제목 키워드 기반
RECONSIDER: Re-Ranking using Span-Focused Cross-Attention for Open Domain Question Answering
State-of-the-art Machine Reading Comprehension (MRC) models for Open-domain Question Answering (QA) are typically trained for span selection using distantly supervised positive examples and heuristically retrieved negati…
Machine Reading ComprehensionNatural QuestionsOpen-Domain Question AnsweringQuestion Answering+3Towards a Better Tradeoff between Effectiveness and Efficiency in Pre-Ranking: A Learnable Feature Selection based Approach
In real-world search, recommendation, and advertising systems, the multi-stage ranking architecture is commonly adopted. Such architecture usually consists of matching, pre-ranking, ranking, and re-ranking stages. In the…
feature selectionRe-RankingA Two-Stage Adaptation of Large Language Models for Text Ranking
Text ranking is a critical task in information retrieval. Recent advances in pre-trained language models (PLMs), especially large language models (LLMs), present new opportunities for applying them to text ranking. While…
DecoderInformation RetrievalRetrievalThoughts on Architecture
The term architecture has evolved considerably from its original Greek roots and its application to buildings and computers to its more recent manifestation for minds. This article considers lessons from this history, in…
InsertRank: LLMs can reason over BM25 scores to Improve Listwise Reranking
Large Language Models (LLMs) have demonstrated significant strides across various information retrieval tasks, particularly as rerankers, owing to their strong generalization and knowledge-transfer capabilities acquired …
Information RetrievalRerankingRetrievalSemantic Similarity+2