SketchZooms: Deep multi-view descriptors for matching line drawings
Finding point-wise correspondences between images is a long-standing problem in image analysis. This becomes particularly challenging for sketch images, due to the varying nature of human drawing style, projection distortions and viewport changes. In this paper we present the first attempt to obtain a learned descriptor for dense registration in line drawings. Based on recent deep learning techniques for corresponding photographs, we designed descriptors to locally match image pairs where the object of interest belongs to the same semantic category, yet still differ drastically in shape, form, and projection angle. To this end, we have specifically crafted a data set of synthetic sketches using non-photorealistic rendering over a large collection of part-based registered 3D models. After training, a neural network generates descriptors for every pixel in an input image, which are shown to generalize correctly in unseen sketches hand-drawn by humans. We evaluate our method against a baseline of correspondences data collected from expert designers, in addition to comparisons with other descriptors that have been proven effective in sketches. Code, data and further resources will be publicly released by the time of publication.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Descriptor-Free Multi-View Region Matching for Instance-Wise 3D Reconstruction
This paper proposes a multi-view extension of instance segmentation without relying on texture or shape descriptor matching. Multi-view instance segmentation becomes challenging for scenes with repetitive textures and sh…
3D ReconstructionInstance SegmentationSegmentationSemantic SegmentationLearning and Matching Multi-View Descriptors for Registration of Point Clouds
Critical to the registration of point clouds is the establishment of a set of accurate correspondences between points in 3D space. The correspondence problem is generally addressed by the design of discriminative 3D loca…
Online Invariance Selection for Local Feature Descriptors
To be invariant, or not to be invariant: that is the question formulated in this work about local descriptors. A limitation of current feature descriptors is the trade-off between generalization and discriminative power:…
DisentanglementHomography EstimationComparative Evaluation of Hand-Crafted and Learned Local Features
Matching local image descriptors is a key step in many computer vision applications. For more than a decade,hand-crafted descriptors such as SIFT have been used for this task. Recently, multiple new descriptors learned f…
Image RetrievalRetrievalImproved Descriptors for Patch Matching and Reconstruction
We propose a convolutional neural network (ConvNet) based approach for learning local image descriptors which can be used for significantly improved patch matching and 3D reconstructions. A multi-resolution ConvNet is us…
3D ReconstructionPatch Matching