Detect-and-describe: Joint learning framework for detection and description of objects
Traditional object detection answers two questions; "what" (what the object is?) and "where" (where the object is?). "what" part of the object detection can be fine-grained further i.e. "what type", "what shape" and "what material" etc. This results in the shifting of the object detection tasks to the object description paradigm. Describing an object provides additional detail that enables us to understand the characteristics and attributes of the object ("plastic boat" not just boat, "glass bottle" not just bottle). This additional information can implicitly be used to gain insight into unseen objects (e.g. unknown object is "metallic", "has wheels"), which is not possible in traditional object detection. In this paper, we present a new approach to simultaneously detect objects and infer their attributes, we call it Detect and Describe (DaD) framework. DaD is a deep learning-based approach that extends object detection to object attribute prediction as well. We train our model on aPascal train set and evaluate our approach on aPascal test set. We achieve 97.0% in Area Under the Receiver Operating Characteristic Curve (AUC) for object attributes prediction on aPascal test set. We also show qualitative results for object attribute prediction on unseen objects, which demonstrate the effectiveness of our approach for describing unknown objects.
Code (1)
Tasks
AttributeObjectobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
R2D2: Repeatable and Reliable Detector and Descriptor
Interest point detection and local feature description are fundamental steps in many computer vision applications. Classical methods for these tasks are based on a detect-then-describe paradigm where separate handcrafted…
Interest Point DetectionKeypoint DetectionMetric LearningR2D2: Reliable and Repeatable Detector and Descriptor
Interest point detection and local feature description are fundamental steps in many computer vision applications. Classical approaches are based on a detect-then-describe paradigm where separate handcrafted methods are …
Atari GamesCamera LocalizationImage MatchingInterest Point Detection+2Decoupling Makes Weakly Supervised Local Feature Better
Weakly supervised learning can help local feature methods to overcome the obstacle of acquiring a large-scale dataset with densely labeled correspondences. However, since weak supervision cannot distinguish the losses ca…
Camera LocalizationImage MatchingWeakly-supervised LearningDescribed Object Detection: Liberating Object Detection with Flexible Expressions
Detecting objects based on language information is a popular task that includes Open-Vocabulary object Detection (OVD) and Referring Expression Comprehension (REC). In this paper, we advance them to a more practical sett…
Binary ClassificationDescribed Object DetectionObjectobject-detection+5Rethinking Low-level Features for Interest Point Detection and Description
Although great efforts have been made for interest point detection and description, the current learning-based methods that use high-level features from the higher layers of Convolutional Neural Networks (CNN) do not com…
Homography EstimationInterest Point DetectionSelf-Supervised Learning