Sparse Meets Dense: A Hybrid Approach to Enhance Scientific Document Retrieval
Traditional information retrieval is based on sparse bag-of-words vector representations of documents and queries. More recent deep-learning approaches have used dense embeddings learned using a transformer-based large language model. We show that on a classic benchmark on scientific document retrieval in the medical domain of cystic fibrosis, that both of these models perform roughly equivalently. Notably, dense vectors from the state-of-the-art SPECTER2 model do not significantly enhance performance. However, a hybrid model that we propose combining these methods yields significantly better results, underscoring the merits of integrating classical and contemporary deep learning techniques in information retrieval in the domain of specialized scientific documents.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningInformation RetrievalLanguage ModelingLanguage ModellingLarge Language ModelRetrievalSimilar Papers 제목 키워드 기반
RecIS: Sparse to Dense, A Unified Training Framework for Recommendation Models
In this paper, we propose RecIS, a unified Sparse-Dense training framework designed to achieve two primary goals: 1. Unified Framework To create a Unified sparse-dense training framework based on the PyTorch ecosystem th…
Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations
Generative models have recently gained attention in recommendation systems by directly predicting item identifiers from user interaction sequences. However, existing methods suffer from significant information loss due t…
QuantizationRecommendation SystemsRetrievalHybrid Autoregressive Inference for Scalable Multi-hop Explanation Regeneration
Regenerating natural language explanations in the scientific domain has been proposed as a benchmark to evaluate complex multi-hop and explainable inference. In this context, large language models can achieve state-of-th…
Multi-hop Question AnsweringNatural Language InferenceQuestion AnsweringAgentic Hybrid RAG for Evidence-Grounded Muon Collider Analysis
Muon collider research spans accelerator physics, detector instrumentation, and high-energy phenomenology, with relevant evidence scattered across a rapidly expanding and heterogeneous body of scientific literature. As h…
Semantic RetrievalQuestion AnsweringAnswer GenerationHybrid Retrieval for COVID-19 Literature: Comparing Rank Fusion and Projection Fusion with Diversity Reranking
We present a hybrid retrieval system for COVID-19 scientific literature, evaluated on the TREC-COVID benchmark (171,332 papers, 50 expert queries). The system implements six retrieval configurations spanning sparse (SPLA…