Neural Best-Buddies: Sparse Cross-Domain Correspondence
Correspondence between images is a fundamental problem in computer vision, with a variety of graphics applications. This paper presents a novel method for sparse cross-domain correspondence. Our method is designed for pairs of images where the main objects of interest may belong to different semantic categories and differ drastically in shape and appearance, yet still contain semantically related or geometrically similar parts. Our approach operates on hierarchies of deep features, extracted from the input images by a pre-trained CNN. Specifically, starting from the coarsest layer in both hierarchies, we search for Neural Best Buddies (NBB): pairs of neurons that are mutual nearest neighbors. The key idea is then to percolate NBBs through the hierarchy, while narrowing down the search regions at each level and retaining only NBBs with significant activations. Furthermore, in order to overcome differences in appearance, each pair of search regions is transformed into a common appearance. We evaluate our method via a user study, in addition to comparisons with alternative correspondence approaches. The usefulness of our method is demonstrated using a variety of graphics applications, including cross-domain image alignment, creation of hybrid images, automatic image morphing, and more.
Code (2)
Tasks
Image MorphingSimilar Papers 제목 키워드 기반
Best Segmentation Buddies for Image-Shape Correspondence
Finding correspondences is a fundamental and extensively researched problem in computer vision and graphics. In this work, we examine the underexplored task of estimating segmentation-to-segmentation correspondence betwe…
Image SegmentationBest-Buddies Tracking
Best-Buddies Tracking (BBT) applies the Best-Buddies Similarity measure (BBS) to the problem of model-free online tracking. BBS was introduced as a similarity measure between two point sets and was shown to be very effec…
Template MatchingBest-Buddies Similarity for Robust Template Matching
We propose a novel method for template matching in unconstrained environments. Its essence is the Best Buddies Similarity (BBS), a useful, robust, and parameter-free similarity measure between two sets of points. BBS is …
Template MatchingBest-Buddies Similarity - Robust Template Matching using Mutual Nearest Neighbors
We propose a novel method for template matching in unconstrained environments. Its essence is the Best-Buddies Similarity (BBS), a useful, robust, and parameter-free similarity measure between two sets of points. BBS is …
Template MatchingBest Buddies Registration for Point Clouds
We propose new, and robust, loss functions for the point cloud registration problem. Our loss functions are inspired by the Best Buddies Similarity (BBS) measure that counts the number of mutual nearest neighbors between…
Point Cloud RegistrationTemplate Matching