Human Correspondence Consensus for 3D Object Semantic Understanding
Semantic understanding of 3D objects is crucial in many applications such as object manipulation. However, it is hard to give a universal definition of point-level semantics that everyone would agree on. We observe that people have a consensus on semantic correspondences between two areas from different objects, but are less certain about the exact semantic meaning of each area. Therefore, we argue that by providing human labeled correspondences between different objects from the same category instead of explicit semantic labels, one can recover rich semantic information of an object. In this paper, we introduce a new dataset named CorresPondenceNet. Based on this dataset, we are able to learn dense semantic embeddings with a novel geodesic consistency loss. Accordingly, several state-of-the-art networks are evaluated on this correspondence benchmark. We further show that CorresPondenceNet could not only boost fine-grained understanding of heterogeneous objects but also cross-object registration and partial object matching.
Code (1)
Tasks
3D Feature Matching3D Point Cloud MatchingObjectSimilar Papers 제목 키워드 기반
Dissecting Generalized Category Discovery: Multiplex Consensus under Self-Deconstruction
Human perceptual systems excel at inducing and recognizing objects across both known and novel categories, a capability far beyond current machine learning frameworks. While generalized category discovery (GCD) aims to b…
Semantic-Aware Fine-Grained Correspondence
Establishing visual correspondence across images is a challenging and essential task. Recently, an influx of self-supervised methods have been proposed to better learn representations for visual correspondence. However, …
Pose TrackingSelf-Supervised LearningSemantic correspondenceSemantic Segmentation+2SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models
Measuring structured object understanding in vision foundation models remains challenging due to inconsistent evaluation protocols and limited part-level supervision. Semantic correspondence (SC) evaluates this capabilit…
Semantic correspondence3D Pose EstimationImage MatchingCategory-Level 3D Correspondence in Camera Space via Morphable Object Priors
Understanding 3D objects from images is fundamental to robotics and AR/VR applications. While recent work has made progress in category-level pose estimation, current representations fail to capture the fine-grained sema…
Pose EstimationRobo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation
Enabling robotic manipulation that generalizes to out-of-distribution scenes is a crucial step toward open-world embodied intelligence. For human beings, this ability is rooted in the understanding of semantic correspond…
Robot ManipulationSemantic correspondence