Diagram Image Retrieval using Sketch-Based Deep Learning and Transfer Learning
Resolution of the complex problem of image retrieval for diagram images has yet to be reached. Deep learning methods continue to excel in the fields of object detection and image classification applied to natural imagery. However, the application of such methodologies applied to binary imagery remains limited due to lack of crucial features such as textures,color and intensity information. This paper presents a deep learning based method for image-based search for binary patent images by taking advantage of existing large natural image repositories for image search and sketch-based methods (Sketches are not identical to diagrams, but they do share some characteristics; for example, both imagery types are gray scale (binary), composed of contours, and are lacking in texture). We begin by using deep learning to generate sketches from natural images for image retrieval and then train a second deep learning model on the sketches. We then use our small set of manually labeled patent diagram images via transfer learning to adapt the image search from sketches of natural images to diagrams. Our experiment results show the effectiveness of deep learning with transfer learning for detecting near-identical copies in patent images and querying similar images based on content.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep Learningimage-classificationImage ClassificationImage Retrievalobject-detectionObject DetectionRetrievalTransfer LearningSimilar Papers 제목 키워드 기반
DiagramRAG: A Lightweight Framework to Retrieve Scientific Diagram for Figure Generation
Scientific diagrams are essential for communicating complex methodologies in academic papers. A natural way for researchers to specify such diagrams is through rough sketches, where text labels, connectors, and spatial a…
Knowledge GraphsSketchAgent: Generating Structured Diagrams from Hand-Drawn Sketches
Hand-drawn sketches are a natural and efficient medium for capturing and conveying ideas. Despite significant advancements in controllable natural image generation, translating freehand sketches into structured, machine-…
Image GenerationYou'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrieval
Two primary input modalities prevail in image retrieval: sketch and text. While text is widely used for inter-category retrieval tasks, sketches have been established as the sole preferred modality for fine-grained image…
AttributeImage RetrievalRetrievalLearning Orbitally Stable Systems for Diagrammatically Teaching
Diagrammatic Teaching is a paradigm for robots to acquire novel skills, whereby the user provides 2D sketches over images of the scene to shape the robot's motion. In this work, we tackle the problem of teaching a robot …
MORPHAdapt and Align to Improve Zero-Shot Sketch-Based Image Retrieval
Zero-shot sketch-based image retrieval (ZS-SBIR) is challenging due to the cross-domain nature of sketches and photos, as well as the semantic gap between seen and unseen image distributions. Previous methods fine-tune p…
Image RetrievalRetrievalSketch-Based Image RetrievalTransfer Learning