Learning to Detect and Retrieve Objects from Unlabeled Videos
Learning an object detector or retrieval requires a large data set with manual annotations. Such data sets are expensive and time consuming to create and therefore difficult to obtain on a large scale. In this work, we propose to exploit the natural correlation in narrations and the visual presence of objects in video, to learn an object detector and retrieval without any manual labeling involved. We pose the problem as weakly supervised learning with noisy labels, and propose a novel object detection paradigm under these constraints. We handle the background rejection by using contrastive samples and confront the high level of label noise with a new clustering score. Our evaluation is based on a set of 11 manually annotated objects in over 5000 frames. We show comparison to a weakly-supervised approach as baseline and provide a strongly labeled upper bound.
Code (1)
Tasks
ClusteringLearning with noisy labelsNovel Object DetectionObjectobject-detectionObject DetectionRetrievalWeakly-supervised LearningSimilar Papers 제목 키워드 기반
MOD-UV: Learning Mobile Object Detectors from Unlabeled Videos
Embodied agents must detect and localize objects of interest, e.g. traffic participants for self-driving cars. Supervision in the form of bounding boxes for this task is extremely expensive. As such, prior work has looke…
Motion SegmentationObjectobject-detectionObject Detection+5Two Video Data Sets for Tracking and Retrieval of Out of Distribution Objects
In this work we present two video test data sets for the novel computer vision (CV) task of out of distribution tracking (OOD tracking). Here, OOD objects are understood as objects with a semantic class outside the seman…
Image SegmentationRetrievalSemantic SegmentationA Self-supervised Learning System for Object Detection in Videos Using Random Walks on Graphs
This paper presents a new self-supervised system for learning to detect novel and previously unseen categories of objects in images. The proposed system receives as input several unlabeled videos of scenes containing var…
Clusteringobject-detectionObject DetectionSelf-Supervised Learning+1Object-Level Targeted Selection via Deep Template Matching
Retrieving images with objects that are semantically similar to objects of interest (OOI) in a query image has many practical use cases. A few examples include fixing failures like false negatives/positives of a learned …
Autonomous DrivingImage RetrievalObjectRetrieval+1LogSTOP: Temporal Scores over Prediction Sequences for Matching and Retrieval
Neural models such as YOLO and HuBERT can be used to detect local properties such as objects ("car") and emotions ("angry") in individual frames of videos and audio clips respectively. The likelihood of these detections …
Video Retrieval