Text Embeddings for Retrieval From a Large Knowledge Base
Text embedding representing natural language documents in a semantic vector space can be used for document retrieval using nearest neighbor lookup. In order to study the feasibility of neural models specialized for retrieval in a semantically meaningful way, we suggest the use of the Stanford Question Answering Dataset (SQuAD) in an open-domain question answering context, where the first task is to find paragraphs useful for answering a given question. First, we compare the quality of various text-embedding methods on the performance of retrieval and give an extensive empirical comparison on the performance of various non-augmented base embedding with, and without IDF weighting. Our main results are that by training deep residual neural models, specifically for retrieval purposes, can yield significant gains when it is used to augment existing embeddings. We also establish that deeper models are superior to this task. The best base baseline embeddings augmented by our learned neural approach improves the top-1 paragraph recall of the system by 14%.
Code (0)
등록된 구현이 없습니다.
Tasks
Open-Domain Question AnsweringQuestion AnsweringRetrievalSimilar Papers 제목 키워드 기반
Enhancing Retrieval-Augmented Generation with Topic-Enriched Embeddings: A Hybrid Approach Integrating Traditional NLP Techniques
Retrieval-augmented generation (RAG) systems rely on accurate document retrieval to ground large language models (LLMs) in external knowledge, yet retrieval quality often degrades in corpora where topics overlap and them…
Dimensionality ReductionEmbedding-Informed Adaptive Retrieval-Augmented Generation of Large Language Models
Retrieval-augmented large language models (LLMs) have been remarkably competent in various NLP tasks. However, it was observed by previous works that retrieval is not always helpful, especially when the LLM is already kn…
Continual LearningRetrievalRetrieval-augmented GenerationEffective Distributed Representations for Academic Expert Search
Expert search aims to find and rank experts based on a user's query. In academia, retrieving experts is an efficient way to navigate through a large amount of academic knowledge. Here, we study how different distributed …
NavigateRetrievalSentenceSentence SimilarityA Multi-Resolution Word Embedding for Document Retrieval from Large Unstructured Knowledge Bases
Deep language models learning a hierarchical representation proved to be a powerful tool for natural language processing, text mining and information retrieval. However, representations that perform well for retrieval mu…
Information RetrievalOpen-Domain Question AnsweringQuestion AnsweringRetrieval+1Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answering
Knowledge-based Visual Question Answering (KB-VQA) requires VQA systems to utilize knowledge from external knowledge bases to answer visually-grounded questions. Retrieval-Augmented Visual Question Answering (RA-VQA), a …
Image to textPassage RetrievalQuestion AnsweringRetrieval+2