Enhancing Multi-Image Question Answering via Submodular Subset Selection
Large multimodal models (LMMs) have achieved high performance in vision-language tasks involving single image but they struggle when presented with a collection of multiple images (Multiple Image Question Answering scenario). These tasks, which involve reasoning over large number of images, present issues in scalability (with increasing number of images) and retrieval performance. In this work, we propose an enhancement for retriever framework introduced in MIRAGE model using submodular subset selection techniques. Our method leverages query-aware submodular functions, such as GraphCut, to pre-select a subset of semantically relevant images before main retrieval component. We demonstrate that using anchor-based queries and augmenting the data improves submodular-retriever pipeline effectiveness, particularly in large haystack sizes.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringRetrievalVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Submodular Minimax Optimization: Finding Effective Sets
Despite the rich existing literature about minimax optimization in continuous settings, only very partial results of this kind have been obtained for combinatorial settings. In this paper, we fill this gap by providing a…
dialog state trackingPrompt EngineeringQuestion AnsweringDIEM: Decomposition-Integration Enhancing Multimodal Insights
In image question answering due to the abundant and sometimes redundant information precisely matching and integrating the information from both text and images is a challenge. In this paper we propose the Decomposit…
MM-VetQuestion AnsweringVisual Question Answering (VQA)SlideVQA: A Dataset for Document Visual Question Answering on Multiple Images
Visual question answering on document images that contain textual, visual, and layout information, called document VQA, has received much attention recently. Although many datasets have been proposed for developing docum…
Evidence SelectionQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)LOVA3: Learning to Visual Question Answering, Asking and Assessment
Question answering, asking, and assessment are three innate human traits crucial for understanding the world and acquiring knowledge. By enhancing these capabilities, humans can more effectively utilize data, leading to …
Question AnsweringVisual Question AnsweringAdapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms
Question categorization and expert retrieval methods have been crucial for information organization and accessibility in community question & answering (CQA) platforms. Research in this area, however, has dealt with only…
Community Question AnsweringGeneral ClassificationQuestion AnsweringRepresentation Learning+4