ZSTAD: Zero-Shot Temporal Activity Detection
An integral part of video analysis and surveillance is temporal activity detection, which means to simultaneously recognize and localize activities in long untrimmed videos. Currently, the most effective methods of temporal activity detection are based on deep learning, and they typically perform very well with large scale annotated videos for training. However, these methods are limited in real applications due to the unavailable videos about certain activity classes and the time-consuming data annotation. To solve this challenging problem, we propose a novel task setting called zero-shot temporal activity detection (ZSTAD), where activities that have never been seen in training can still be detected. We design an end-to-end deep network based on R-C3D as the architecture for this solution. The proposed network is optimized with an innovative loss function that considers the embeddings of activity labels and their super-classes while learning the common semantics of seen and unseen activities. Experiments on both the THUMOS14 and the Charades datasets show promising performance in terms of detecting unseen activities.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionActivity DetectionSimilar Papers 제목 키워드 기반
Training-Free Zero-Shot Temporal Action Detection with Vision-Language Models
Existing zero-shot temporal action detection (ZSTAD) methods predominantly use fully supervised or unsupervised strategies to recognize unseen activities. However, these training-based methods are prone to domain shifts …
Action DetectionPseudo LabelTest-time AdaptationTF-CADE: Foreground-Concentrated Text-Video Alignment for Zero-Shot Temporal Action Detection
Zero-Shot Temporal Action Detection (ZSTAD) aims to lo- calize and recognize action instances from unseen action categories in untrimmed videos. Although existing meth- ods have shown effectiveness by advancing architect…
Action DetectionVideo AlignmentSimilarity R-C3D for Few-shot Temporal Activity Detection
Many activities of interest are rare events, with only a few labeled examples available. Therefore models for temporal activity detection which are able to learn from a few examples are desirable. In this paper, we prese…
Action DetectionActivity DetectionZEETAD: Adapting Pretrained Vision-Language Model for Zero-Shot End-to-End Temporal Action Detection
Temporal action detection (TAD) involves the localization and classification of action instances within untrimmed videos. While standard TAD follows fully supervised learning with closed-set setting on large training dat…
Action DetectionClassificationLanguage ModelingLanguage Modelling+2Revisiting Few-shot Activity Detection with Class Similarity Control
Many interesting events in the real world are rare making preannotated machine learning ready videos a rarity in consequence. Thus, temporal activity detection models that are able to learn from a few examples are desira…
Action DetectionActivity DetectionVideo Classification