MARCO: Navigating the Unseen Space of Semantic Correspondence
Recent advances in semantic correspondence rely on dual-encoder architectures, combining DINOv2 with diffusion backbones. While accurate, these billion-parameter models generalize poorly beyond training keypoints, revealing a gap between benchmark performance and real-world usability, where queried points rarely match those seen during training. Building upon DINOv2, we introduce MARCO, a unified model for generalizable correspondence driven by a novel training framework that enhances both fine-grained localization and semantic generalization. By coupling a coarse-to-fine objective that refines spatial precision with a self-distillation framework, which expands sparse supervision beyond annotated regions, our approach transforms a handful of keypoints into dense, semantically coherent correspondences. MARCO sets a new state of the art on SPair-71k, AP-10K, and PF-PASCAL, with gains that amplify at fine-grained localization thresholds (+8.9 PCK@0.01), strongest generalization to unseen keypoints (+5.1, SPair-U) and categories (+4.7, MP-100), while remaining 3x smaller and 10x faster than diffusion-based approaches. Code is available at https://github.com/visinf/MARCO .
Code (0)
등록된 구현이 없습니다.
Tasks
Semantic correspondenceResults from the Paper
| Rank | Task | Dataset | Model | Metrics |
|---|---|---|---|---|
| #45 | Semantic correspondence | SPair-71k | MARCO | PCK: 8.9 |
Similar Papers 제목 키워드 기반
Zero-Shot Point Cloud Segmentation by Semantic-Visual Aware Synthesis
This paper proposes a feature synthesis approach for zero-shot semantic segmentation of 3D point clouds, enabling generalization to previously unseen categories. Given only the class-level semantic information for un…
DiversityPoint Cloud SegmentationSegmentationSemantic Segmentation+1Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondence
Semantic correspondence (SC) aims to establish semantically meaningful matches across different instances of an object category. We illustrate how recent supervised SC methods remain limited in their ability to generaliz…
Depth EstimationMonocular Depth EstimationSemantic correspondenceMatchingPolicy: Correspondence-Aware Policy Enables Cross-Object In-Context Learning
In-context imitation learning enables few-shot policy generalization but struggles to maintain performance on unseen objects and novel scenarios. To address this, we introduce MatchingPolicy, a correspondence-driven fram…
GRAFT: Graph-Based Affordance Transfer via Part Correspondence
Generalizing robotic manipulation to unseen objects remains challenging, as learning-based approaches require many demonstrations and fail in few-shot settings. Prior work transfers affordances through semantic retrieval…
Semantic RetrievalZero-shot Image Recognition Using Relational Matching, Adaptation and Calibration
Zero-shot learning (ZSL) for image classification focuses on recognizing novel categories that have no labeled data available for training. The learning is generally carried out with the help of mid-level semantic descri…
Domain AdaptationGeneral Classificationimage-classificationImage Classification+1