Towards Train-Test Consistency for Semi-supervised Temporal Action Localization
Recently, Weakly-supervised Temporal Action Localization (WTAL) has been densely studied but there is still a large gap between weakly-supervised models and fully-supervised models. It is practical and intuitive to annotate temporal boundaries of a few examples and utilize them to help WTAL models better detect actions. However, the train-test discrepancy of action localization strategy prevents WTAL models from leveraging semi-supervision for further improvement. At training time, attention or multiple instance learning is used to aggregate predictions of each snippet for video-level classification; at test time, they first obtain action score sequences over time, then truncate segments of scores higher than a fixed threshold, and post-process action segments. The inconsistent strategy makes it hard to explicitly supervise the action localization model with temporal boundary annotations at training time. In this paper, we propose a Train-Test Consistent framework, TTC-Loc. In both training and testing time, our TTC-Loc localizes actions by comparing scores of action classes and predicted threshold, which enables it to be trained with semi-supervision. By fixing the train-test discrepancy, our TTC-Loc significantly outperforms the state-of-the-art performance on THUMOS'14, ActivityNet 1.2 and 1.3 when only video-level labels are provided for training. With full annotations of only one video per class and video-level labels for the other videos, our TTC-Loc further boosts the performance and achieves 33.4\% mAP (IoU threshold 0.5) on THUMOS's 14.
Code (0)
등록된 구현이 없습니다.
Tasks
Action LocalizationMultiple Instance LearningTemporal Action LocalizationVideo ClassificationWeakly-supervised Temporal Action LocalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
End-to-End Semi-Supervised Learning for Video Action Detection
In this work, we focus on semi-supervised learning for video action detection which utilizes both labeled as well as unlabeled data. We propose a simple end-to-end consistency based approach which effectively utilizes th…
Action DetectionClassification ConsistencySemantic SegmentationSemi-Supervised Video Action Detection+4Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results
The recently proposed Temporal Ensembling has achieved state-of-the-art results in several semi-supervised learning benchmarks. It maintains an exponential moving average of label predictions on each training example, an…
Semi-Supervised Image ClassificationSemi-Supervised RGBD Semantic SegmentationSemi-Supervised Semantic SegmentationSource Free Object DetectionSemi-supervised Video Semantic Segmentation Using Unreliable Pseudo Labels for PVUW2024
Pixel-level Scene Understanding is one of the fundamental problems in computer vision, which aims at recognizing object classes, masks and semantics of each pixel in the given image. Compared with image scene parsing, vi…
Scene ParsingScene UnderstandingSemantic SegmentationVideo Semantic SegmentationStructured Consistency Loss for semi-supervised semantic segmentation
The consistency loss has played a key role in solving problems in recent studies on semi-supervised learning. Yet extant studies with the consistency loss are limited to its application to classification tasks; extant st…
General ClassificationSegmentationSemantic SegmentationSemi-Supervised Semantic SegmentationC2F-SemiCD: A Coarse-to-Fine Semi-Supervised Change Detection Method Based on Consistency Regularization in High-Resolution Remote Sensing Images
A high-precision feature extraction model is crucial for change detection (CD). In the past, many deep learning-based supervised CD methods learned to recognize change feature patterns from a large number of labelled bi-…
Change DetectionSemi-supervised Change Detection