Exploring Sub-Pseudo Labels for Learning from Weakly-Labeled Web Videos
Learning visual knowledge from massive weakly-labeled web videos has attracted growing research interests thanks to the large corpus of easily accessible video data on the Internet. However, for video action recognition, the action of interest might only exist in arbitrary clips of untrimmed web videos, resulting in high label noises in the temporal space. To address this issue, we introduce a new method for pre-training video action recognition models using queried web videos. Instead of trying to filter out, we propose to convert the potential noises in these queried videos to useful supervision signals by defining the concept of Sub-Pseudo Label (SPL). Specifically, SPL spans out a new set of meaningful "middle ground" label space constructed by extrapolating the original weak labels during video querying and the prior knowledge distilled from a teacher model. Consequently, SPL provides enriched supervision for video models to learn better representations for downstream tasks. We validate the effectiveness of our method on four video action recognition datasets and a weakly-labeled image dataset to study the generalization ability. Experiments show that SPL outperforms several existing pre-training strategies using pseudo-labels and achieves competitive results on HMDB-51 and UCF-101 datasets compared with recent pre-training methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionPseudo LabelTemporal Action LocalizationSimilar Papers 제목 키워드 기반
Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts
Learning visual knowledge from massive weakly-labeled web videos has attracted growing research interests thanks to the large corpus of easily accessible video data on the Internet. However, for video action recognition,…
Action RecognitionPseudo LabelTemporal Action LocalizationTrack and Transfer: Watching Videos to Simulate Strong Human Supervision for Weakly-Supervised Object Detection
The status quo approach to training object detectors requires expensive bounding box annotations. Our framework takes a markedly different direction: we transfer tracked object boxes from weakly-labeled videos to weakly-…
Objectobject-detectionObject DetectionWeakly Supervised Object DetectionWeakly Labeled Sound Event Detection Using Tri-training and Adversarial Learning
This paper considers a semi-supervised learning framework for weakly labeled polyphonic sound event detection problems for the DCASE 2019 challenge's task4 by combining both the tri-training and adversarial learning. The…
Event DetectionSound Event DetectionAdvancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling
The Audio-Visual Video Parsing task aims to identify and temporally localize the events that occur in either or both the audio and visual streams of audible videos. It often performs in a weakly-supervised manner, where …
audio-visual event localizationDenoisingPseudo LabelSemi-Weakly-Supervised Learning of Complex Actions From Instructional Task Videos
We address the problem of action segmentation in instructional task videos with a small number of weakly-labeled training videos and a large number of unlabeled videos, which we refer to as Semi-Weakly-Supervised Lea…
Action SegmentationWeakly-supervised Learning