LoRE: Logit-Ranked Retriever Ensemble for Enhancing Open-Domain Question Answering
Retrieval-based question answering systems often suffer from positional bias, leading to suboptimal answer generation. We propose LoRE (Logit-Ranked Retriever Ensemble), a novel approach that improves answer accuracy and relevance by mitigating positional bias. LoRE employs an ensemble of diverse retrievers, such as BM25 and sentence transformers with FAISS indexing. A key innovation is a logit-based answer ranking algorithm that combines the logit scores from a large language model (LLM), with the retrieval ranks of the passages. Experimental results on NarrativeQA, SQuAD demonstrate that LoRE significantly outperforms existing retrieval-based methods in terms of exact match and F1 scores. On SQuAD, LoRE achieves 14.5\%, 22.83\%, and 14.95\% improvements over the baselines for ROUGE-L, EM, and F1, respectively. Qualitatively, LoRE generates more relevant and accurate answers, especially for complex queries.
Code (0)
등록된 구현이 없습니다.
Tasks
Answer GenerationLanguage ModelingLanguage ModellingLarge Language ModelOpen-Domain Question AnsweringQuestion AnsweringRetrievalSentenceSimilar Papers 제목 키워드 기반
SWAP: Exploiting Second-Ranked Logits for Adversarial Attacks on Time Series
Time series classification (TSC) has emerged as a critical task in various domains, and deep neural models have shown superior performance in TSC tasks. However, these models are vulnerable to adversarial attacks, where …
Time SeriesTime Series ClassificationHybrid and Collaborative Passage Reranking
In passage retrieval system, the initial passage retrieval results may be unsatisfactory, which can be refined by a reranking scheme. Existing solutions to passage reranking focus on enriching the interaction between que…
Passage RerankingPassage RetrievalRerankingRetrievalExp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query Expansion
Large Language Models (LLMs) have shown potential in generating hypothetical documents for query expansion, thereby enhancing information retrieval performance. However, the efficacy of this method is highly dependent on…
Information RetrievalLanguage ModelingLanguage ModellingLarge Language Model+1Beyond Sole Strength: Customized Ensembles for Generalized Vision-Language Models
Fine-tuning pre-trained vision-language models (VLMs), e.g., CLIP, for the open-world generalization has gained increasing popularity due to its practical value. However, performance advancements are limited when relying…
Prompt EngineeringQuantifying Retriever-Generator Alignment in RAG with Local Explanations
Retrieval-Augmented Generation (RAG) systems combine dense retrievers and language models to ground their outputs in external documents. However, the interaction between these components remains opaque, creating challeng…