Three-Stream Joint Network for Zero-Shot Sketch-Based Image Retrieval
The Zero-Shot Sketch-based Image Retrieval (ZS-SBIR) is a challenging task because of the large domain gap between sketches and natural images as well as the semantic inconsistency between seen and unseen categories. Previous literature bridges seen and unseen categories by semantic embedding, which requires prior knowledge of the exact class names and additional extraction efforts. And most works reduce domain gap by mapping sketches and natural images into a common high-level space using constructed sketch-image pairs, which ignore the unpaired information between images and sketches. To address these issues, in this paper, we propose a novel Three-Stream Joint Training Network (3JOIN) for the ZS-SBIR task. To narrow the domain differences between sketches and images, we extract edge maps for natural images and treat them as a bridge between images and sketches, which have similar content to images and similar style to sketches. For exploiting a sufficient combination of sketches, natural images, and edge maps, a novel three-stream joint training network is proposed. In addition, we use a teacher network to extract the implicit semantics of the samples without the aid of other semantics and transfer the learned knowledge to unseen classes. Extensive experiments conducted on two real-world datasets demonstrate the superiority of our proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Image RetrievalRetrievalSketch-Based Image RetrievalSimilar Papers 제목 키워드 기반
Modality-Aware Representation Learning for Zero-shot Sketch-based Image Retrieval
Zero-shot learning offers an efficient solution for a machine learning model to treat unseen categories, avoiding exhaustive data collection. Zero-shot Sketch-based Image Retrieval (ZS-SBIR) simulates real-world scenario…
Image RetrievalRepresentation LearningRetrievalSketch-Based Image Retrieval+1Zero-Shot Sketch-Image Hashing
Recent studies show that large-scale sketch-based image retrieval (SBIR) can be efficiently tackled by cross-modal binary representation learning methods, where Hamming distance matching significantly speeds up the proce…
Image RetrievalRepresentation LearningRetrievalSketch-Based Image RetrievalACNet: Approaching-and-Centralizing Network for Zero-Shot Sketch-Based Image Retrieval
The huge domain gap between sketches and photos and the highly abstract sketch representations pose challenges for sketch-based image retrieval (\underline{SBIR}). The zero-shot sketch-based image retrieval (\underline{Z…
Image RetrievalRetrievalSketch-Based Image RetrievalCross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval
In this paper, we study the problem of zero-shot sketch-based image retrieval (ZS-SBIR). The prior methods tackle the problem in a two-modality setting with only category labels or even no textual information involved. H…
cross-modal alignmentImage RetrievalRetrievalSketch-Based Image Retrieval+1Sketch3T: Test-Time Training for Zero-Shot SBIR
Zero-shot sketch-based image retrieval typically asks for a trained model to be applied as is to unseen categories. In this paper, we question to argue that this setup by definition is not compatible with the inherent ab…
Image RetrievalMeta-LearningRetrievalSketch-Based Image Retrieval