Fine-Grained Zero-Shot Object Detection
Zero-shot object detection (ZSD) aims to leverage semantic descriptions to localize and recognize objects of both seen and unseen classes. Existing ZSD works are mainly coarse-grained object detection, where the classes are visually quite different, thus are relatively easy to distinguish. However, in real life we often have to face fine-grained object detection scenarios, where the classes are too similar to be easily distinguished. For example, detecting different kinds of birds, fishes, and flowers. In this paper, we propose and solve a new problem called Fine-Grained Zero-Shot Object Detection (FG-ZSD for short), which aims to detect objects of different classes with minute differences in details under the ZSD paradigm. We develop an effective method called MSHC for the FG-ZSD task, which is based on an improved two-stage detector and employs a multi-level semantics-aware embedding alignment loss, ensuring tight coupling between the visual and semantic spaces. Considering that existing ZSD datasets are not suitable for the new FG-ZSD task, we build the first FG-ZSD benchmark dataset FGZSD-Birds, which contains 148,820 images falling into 36 orders, 140 families, 579 genera and 1432 species. Extensive experiments on FGZSD-Birds show that our method outperforms existing ZSD models.
Code (0)
등록된 구현이 없습니다.
Tasks
Zero-Shot Object DetectionSimilar Papers 제목 키워드 기반
A Zero-Shot Claim Detection Framework Using Question Answering
In recent years, there has been an increasing interest in claim detection as an important building block for misinformation detection. This involves detecting more fine-grained attributes relating to the claim, such as t…
MisinformationObjectobject-detectionObject Detection+1Locality-Aware Zero-Shot Human-Object Interaction Detection
Recent methods for zero-shot Human-Object Interaction (HOI) detection typically leverage the generalization ability of large Vision-Language Model (VLM), i.e., CLIP, on unseen categories, showing impressive results on va…
Human-Object Interaction DetectionObjectZero-Shot Human-Object Interaction DetectionCLIP4HOI: Towards Adapting CLIP for Practical Zero-Shot HOI Detection
Zero-shot Human-Object Interaction (HOI) detection aims to identify both seen and unseen HOI categories. A strong zero-shot HOI detector is supposed to be not only capable of discriminating novel interactions but also ro…
Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognition
Zero-shot object recognition or zero-shot learning aims to transfer the object recognition ability among the semantically related categories, such as fine-grained animal or bird species. However, the images of different …
Generalized Zero-Shot LearningObjectObject RecognitionZero-Shot LearningReal Classification by Description: Extending CLIP's Limits of Part Attributes Recognition
In this study, we define and tackle zero shot "real" classification by description, a novel task that evaluates the ability of Vision-Language Models (VLMs) like CLIP to classify objects based solely on descriptive attri…
AttributeDescriptiveObjectObject Recognition+1