Multi-granularity Association Learning Framework for on-the-fly Fine-Grained Sketch-based Image Retrieval
Fine-grained sketch-based image retrieval (FG-SBIR) addresses the problem of retrieving a particular photo in a given query sketch. However, its widespread applicability is limited by the fact that it is difficult to draw a complete sketch for most people, and the drawing process often takes time. In this study, we aim to retrieve the target photo with the least number of strokes possible (incomplete sketch), named on-the-fly FG-SBIR (Bhunia et al. 2020), which starts retrieving at each stroke as soon as the drawing begins. We consider that there is a significant correlation among these incomplete sketches in the sketch drawing episode of each photo. To learn more efficient joint embedding space shared between the photo and its incomplete sketches, we propose a multi-granularity association learning framework that further optimizes the embedding space of all incomplete sketches. Specifically, based on the integrity of the sketch, we can divide a complete sketch episode into several stages, each of which corresponds to a simple linear mapping layer. Moreover, our framework guides the vector space representation of the current sketch to approximate that of its later sketches to realize the retrieval performance of the sketch with fewer strokes to approach that of the sketch with more strokes. In the experiments, we proposed more realistic challenges, and our method achieved superior early retrieval efficiency over the state-of-the-art methods and alternative baselines on two publicly available fine-grained sketch retrieval datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Image RetrievalRetrievalSketch-Based Image RetrievalSimilar Papers 제목 키워드 기반
U-shaped Multi-granularity Learning for Vision-Language Models
The prompt learning paradigm for vision-language models is effective yet faces a granularity dilemma: global prompts lack fine-grained semantic awareness, while local prompts ignore contextual associations, limiting cros…
Cross-Granularity Hypergraph Retrieval-Augmented Generation for Multi-hop Question Answering
Multi-hop question answering (MHQA) requires integrating knowledge scattered across multiple passages to derive the correct answer. Traditional retrieval-augmented generation (RAG) methods primarily focus on coarse-grain…
Multi-hop Question AnsweringSemantic SimilarityAnswer GenerationKnowledge GraphsMulti-Granularity Framework for Unsupervised Representation Learning of Time Series
Representation learning plays a critical role in the analysis of time series data and has high practical value across a wide range of applications. including trend analysis, time series data retrieval and forecasting. In…
Decision MakingRepresentation LearningRetrievalTime SeriesEnGraf-Net: Multiple Granularity Branch Network with Fine-Coarse Graft Grained for Classification Task
Fine-Grained classification models can expressly focus on the relevant details useful to distinguish highly similar classes typically when the intra-class variance is high and the inter-class variance is low given a data…
ClassificationFine-Grained Image ClassificationImage ClassificationEnGraf-Net: Multiple Granularity Branch Network with Fine-Coarse Graft Grained for Classification Task
Fine-grained classification models are designed to focus on the relevant details necessary to distinguish highly similar classes, particularly when intra-class variance is high and inter-class variance is low. Most exist…