Efficient Media Retrieval from Non-Cooperative Queries
Text is ubiquitous in the artificial world and easily attainable when it comes to book title and author names. Using the images from the book cover set from the Stanford Mobile Visual Search dataset and additional book covers and metadata from openlibrary.org, we construct a large scale book cover retrieval dataset, complete with 100K distractor covers and title and author strings for each. Because our query images are poorly conditioned for clean text extraction, we propose a method for extracting a matching noisy and erroneous OCR readings and matching it against clean author and book title strings in a standard document look-up problem setup. Finally, we demonstrate how to use this text-matching as a feature in conjunction with popular retrieval features such as VLAD using a simple learning setup to achieve significant improvements in retrieval accuracy over that of either VLAD or the text alone.
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Character Recognition (OCR)RetrievalText MatchingSimilar Papers 제목 키워드 기반
Cross-Media Similarity Evaluation for Web Image Retrieval in the Wild
In order to retrieve unlabeled images by textual queries, cross-media similarity computation is a key ingredient. Although novel methods are continuously introduced, little has been done to evaluate these methods togethe…
Image RetrievalRetrievalEnd-to-end Knowledge Retrieval with Multi-modal Queries
We investigate knowledge retrieval with multi-modal queries, i.e. queries containing information split across image and text inputs, a challenging task that differs from previous work on cross-modal retrieval. We curate …
BenchmarkingCross-Modal RetrievalRetrievalVisual Question Answering (VQA)Contextual Media Retrieval Using Natural Language Queries
The widespread integration of cameras in hand-held and head-worn devices as well as the ability to share content online enables a large and diverse visual capture of the world that millions of users build up collectively…
Natural Language QueriesRetrievalShapley Value Computation in Ontology-Mediated Query Answering
The Shapley value, originally introduced in cooperative game theory for wealth distribution, has found use in KR and databases for the purpose of assigning scores to formulas and database tuples based upon their contribu…
Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
Retrieval-augmented generation (RAG) encounters challenges when addressing complex queries, particularly multi-hop questions. While several methods tackle multi-hop queries by iteratively generating internal queries and …
Multi-hop Question AnsweringQuestion AnsweringRAGRetrieval+1