Structured Query-Based Image Retrieval Using Scene Graphs
A structured query can capture the complexity of object interactions (e.g. 'woman rides motorcycle') unlike single objects (e.g. 'woman' or 'motorcycle'). Retrieval using structured queries therefore is much more useful than single object retrieval, but a much more challenging problem. In this paper we present a method which uses scene graph embeddings as the basis for an approach to image retrieval. We examine how visual relationships, derived from scene graphs, can be used as structured queries. The visual relationships are directed subgraphs of the scene graph with a subject and object as nodes connected by a predicate relationship. Notably, we are able to achieve high recall even on low to medium frequency objects found in the long-tailed COCO-Stuff dataset, and find that adding a visual relationship-inspired loss boosts our recall by 10% in the best case.
Code (0)
등록된 구현이 없습니다.
Tasks
Image RetrievalObjectRetrievalSimilar Papers 제목 키워드 기반
Compact Scene Graphs for Layout Composition and Patch Retrieval
Structured representations such as scene graphs serve as an efficient and compact representation that can be used for downstream rendering or retrieval tasks. However, existing efforts to generate realistic images from s…
Image GenerationRetrievalResponse-G1: Explicit Scene Graph Modeling for Proactive Streaming Video Understanding
Proactive streaming video understanding requires Video-LLMs to decide when to respond as a video unfolds, a task where existing methods often fall short due to their implicit, query-agnostic modeling of visual evidence. …
Scene Graph GenerationImage-to-Image Retrieval by Learning Similarity between Scene Graphs
As a scene graph compactly summarizes the high-level content of an image in a structured and symbolic manner, the similarity between scene graphs of two images reflects the relevance of their contents. Based on this idea…
Graph SimilarityImage RetrievalRetrievalSentence+1Approximate Query Matching for Image Retrieval
Traditional image recognition involves identifying the key object in a portrait-type image with a single object focus (ILSVRC, AlexNet, and VGG). More recent approaches consider dense image recognition - segmenting an im…
Graph GenerationImage RetrievalObjectRetrieval+2SupScene: Scene-Structured Overlap Supervision for Image Retrieval in Unconstrained SfM
Image retrieval is a critical step for reducing the quadratic cost of image matching in unconstrained Structure-from-Motion (SfM). Unlike generic image retrieval, however, the relevant goal of SfM is to identify geometri…
Image RetrievalImage Matching