Query Generation for Multimodal Documents
This paper studies the problem of generatinglikely queries for multimodal documents withimages. Our application scenario is enablingefficient {``}first-stage retrieval{''} of relevant doc-uments, by attaching generated queries to doc-uments before indexing. We can then indexthis expanded text to efficiently narrow downto candidate matches using inverted index, sothat expensive reranking can follow. Our eval-uation results show that our proposed multi-modal representation meaningfully improvesrelevance ranking.More importantly, ourframework can achieve the state of the art inthe first stage retrieval scenarios
Code (0)
등록된 구현이 없습니다.
Tasks
RerankingRetrievalSimilar Papers 제목 키워드 기반
MARA: A Multimodal Adaptive Retrieval-Augmented Framework for Document Question Answering
Retrieval-based multimodal document QA aims to identify and integrate relevant information from visually rich documents with complex multimodal structures. While retrieval-augmented generation (RAG) has shown strong perf…
Question AnsweringSelf-adaptive Multimodal Retrieval-Augmented Generation
Traditional Retrieval-Augmented Generation (RAG) methods are limited by their reliance on a fixed number of retrieved documents, often resulting in incomplete or noisy information that undermines task performance. Althou…
Image CaptioningRAGResponse GenerationRetrieval+1MLDocRAG: Multimodal Long-Context Document Retrieval Augmented Generation
Understanding multimodal long-context documents that comprise multimodal chunks such as paragraphs, figures, and tables is challenging due to (1) cross-modal heterogeneity to localize relevant information across modaliti…
Long-Context UnderstandingQuestion AnsweringZero-Shot Document Understanding using Pseudo Table of Contents-Guided Retrieval-Augmented Generation
Understanding complex multimodal documents remains challenging due to their structural inconsistencies and limited training data availability. We introduce \textit{DocsRay}, a training-free document understanding system …
mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQA
Retrieval-Augmented Generation (RAG) has emerged as an effective paradigm for expanding the knowledge capacity of Multimodal Large Language Models (MLLMs) by incorporating external knowledge sources into the generation p…
Visual Question AnsweringKnowledge Graphs