Integrating Visual and Semantic Similarity Using Hierarchies for Image Retrieval
Most of the research in content-based image retrieval (CBIR) focus on developing robust feature representations that can effectively retrieve instances from a database of images that are visually similar to a query. However, the retrieved images sometimes contain results that are not semantically related to the query. To address this, we propose a method for CBIR that captures both visual and semantic similarity using a visual hierarchy. The hierarchy is constructed by merging classes with overlapping features in the latent space of a deep neural network trained for classification, assuming that overlapping classes share high visual and semantic similarities. Finally, the constructed hierarchy is integrated into the distance calculation metric for similarity search. Experiments on standard datasets: CUB-200-2011 and CIFAR100, and a real-life use case using diatom microscopy images show that our method achieves superior performance compared to the existing methods on image retrieval.
Code (1)
Tasks
Content-Based Image RetrievalImage RetrievalRetrievalSemantic SimilaritySemantic Textual SimilarityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning Visual Hierarchies with Hyperbolic Embeddings
Structuring latent representations in a hierarchical manner enables models to learn patterns at multiple levels of abstraction. However, most prevalent image understanding models focus on visual similarity, and learning …
Image RetrievalRetrievalHyperPath: Knowledge-Guided Hyperbolic Semantic Hierarchy Modeling for WSI Analysis
Pathology is essential for cancer diagnosis, with multiple instance learning (MIL) widely used for whole slide image (WSI) analysis. WSIs exhibit a natural hierarchy -- patches, regions, and slides -- with distinct seman…
cross-modal alignmentMultiple Instance LearningHierarchy-based Image Embeddings for Semantic Image Retrieval
Deep neural networks trained for classification have been found to learn powerful image representations, which are also often used for other tasks such as comparing images w.r.t. their visual similarity. However, visual …
Few-Shot LearningImage RetrievalNovelty DetectionRetrieval+3Combinaison d'information visuelle, conceptuelle, et contextuelle pour la construction automatique de hierarchies semantiques adaptees a l'annotation d'images
This paper proposes a new methodology to automatically build semantic hierarchies suitable for image annotation and classification. The building of the hierarchy is based on a new measure of semantic similarity. The prop…
ClassificationGeneral ClassificationSemantic SimilaritySemantic Textual SimilarityLearning with Hierarchical Complement Objective
Label hierarchies widely exist in many vision-related problems, ranging from explicit label hierarchies existed in image classification to latent label hierarchies existed in semantic segmentation. Nevertheless, state-of…
General Classificationimage-classificationImage ClassificationSegmentation+1