Adaptive Mutual Supervision for Weakly-Supervised Temporal Action Localization
Weakly-supervised temporal action localization aims to localize actions in untrimmed videos with only video-level action category labels. Most of previous methods ignore the incompleteness issue of Class Activation Sequences (CAS), suffering from trivial localization results. To solve this issue, we introduce an adaptive mutual supervision framework (AMS) with two branches, where the base branch adopts CAS to localize the most discriminative action regions, while the supplementary branch localizes the less discriminative action regions through a novel adaptive sampler. The adaptive sampler dynamically updates the input of the supplementary branch with a sampling weight sequence negatively correlated with the CAS from the base branch, thereby prompting the supplementary branch to localize the action regions underestimated by the base branch. To promote mutual enhancement between these two branches, we construct mutual location supervision. Each branch leverages location pseudo-labels generated from the other branch as localization supervision. By alternately optimizing the two branches in multiple iterations, we progressively complete action regions. Extensive experiments on THUMOS14 and ActivityNet1.2 demonstrate that the proposed AMS method significantly outperforms the state-of-the-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Action LocalizationTemporal Action LocalizationWeakly Supervised Action LocalizationWeakly-supervised Temporal Action LocalizationSimilar Papers 제목 키워드 기반
Weakly-Supervised Spatio-Temporal Anomaly Detection in Surveillance Video
In this paper, we introduce a novel task, referred to as Weakly-Supervised Spatio-Temporal Anomaly Detection (WSSTAD) in surveillance video. Specifically, given an untrimmed video, WSSTAD aims to localize a spatio-tempor…
Anomaly DetectionWeakly Supervised Temporal Sentence Grounding With Uncertainty-Guided Self-Training
The task of weakly supervised temporal sentence grounding aims at finding the corresponding temporal moments of a language description in the video, given video-language correspondence only at video-level. Most exist…
Data AugmentationSentenceTemporal Sentence GroundingWeakly-supervised LearningCLASP: Cross-modal Salient Anchor-based Semantic Propagation for Weakly-supervised Dense Audio-Visual Event Localization
The Dense Audio-Visual Event Localization (DAVEL) task aims to temporally localize events in untrimmed videos that occur simultaneously in both the audio and visual modalities. This paper explores DAVEL under a new and m…
audio-visual event localizationD2-Net: Weakly-Supervised Action Localization via Discriminative Embeddings and Denoised Activations
This work proposes a weakly-supervised temporal action localization framework, called D2-Net, which strives to temporally localize actions using video-level supervision. Our main contribution is the introduction of a nov…
Action LocalizationDenoisingTemporal Action LocalizationWeakly Supervised Action Localization+1Branches Mutual Promotion for End-to-End Weakly Supervised Semantic Segmentation
End-to-end weakly supervised semantic segmentation aims at optimizing a segmentation model in a single-stage training process based on only image annotations. Existing methods adopt an online-trained classification branc…
ClassificationSegmentationSemantic SegmentationWeakly supervised segmentation+2