ShapeY: Measuring Shape Recognition Capacity Using Nearest Neighbor Matching
Object recognition in humans depends primarily on shape cues. We have developed a new approach to measuring the shape recognition performance of a vision system based on nearest neighbor view matching within the system's embedding space. Our performance benchmark, ShapeY, allows for precise control of task difficulty, by enforcing that view matching span a specified degree of 3D viewpoint change and/or appearance change. As a first test case we measured the performance of ResNet50 pre-trained on ImageNet. Matching error rates were high. For example, a 27 degree change in object pitch led ResNet50 to match the incorrect object 45% of the time. Appearance changes were also highly disruptive. Examination of false matches indicates that ResNet50's embedding space is severely "tangled". These findings suggest ShapeY can be a useful tool for charting the progress of artificial vision systems towards human-level shape recognition capabilities.
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectObject RecognitionSimilar Papers 제목 키워드 기반
ShapeY: A Principled Framework for Measuring Shape Recognition Capacity via Nearest-Neighbor Matching
Object recognition (OR) in humans relies heavily on shape cues and the ability to recognize objects across varying 3D viewpoints. Unlike humans, deep networks often rely on non-shape cues such as texture and background, …
Object RecognitionDisentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition
This paper proposes an encoder-decoder network to disentangle shape features during 3D face reconstruction from single 2D images, such that the tasks of reconstructing accurate 3D face shapes and learning discriminative …
3D Face ReconstructionDecoderFace IdentificationFace Recognition+1Towards Applying the OPRA Theory to Shape Similarity
The motivation for using qualitative shape descriptions is as follows: qualitative shape descriptions can implicitly act as a schema for measuring the similarity of shapes, which has the potential to be cognitively adequ…
Comprehensive Equity Index (CEI): Definition and Application to Bias Evaluation in Biometrics
We present a novel metric designed, among other applications, to quantify biased behaviors of machine learning models. As its core, the metric consists of a new similarity metric between score distributions that balances…
Face RecognitionFairnessThe Platonic Universe: Do Foundation Models See the Same Sky?
We test the Platonic Representation Hypothesis (PRH) in astronomy by measuring representational convergence across a range of foundation models trained on different data types. Using spectroscopic and imaging observation…