Action Unit Memory Network for Weakly Supervised Temporal Action Localization
Weakly supervised temporal action localization aims to detect and localize actions in untrimmed videos with only video-level labels during training. However, without frame-level annotations, it is challenging to achieve localization completeness and relieve background interference. In this paper, we present an Action Unit Memory Network (AUMN) for weakly supervised temporal action localization, which can mitigate the above two challenges by learning an action unit memory bank. In the proposed AUMN, two attention modules are designed to update the memory bank adaptively and learn action units specific classifiers. Furthermore, three effective mechanisms (diversity, homogeneity and sparsity) are designed to guide the updating of the memory network. To the best of our knowledge, this is the first work to explicitly model the action units with a memory network. Extensive experimental results on two standard benchmarks (THUMOS14 and ActivityNet) demonstrate that our AUMN performs favorably against state-of-the-art methods. Specifically, the average mAP of IoU thresholds from 0.1 to 0.5 on the THUMOS14 dataset is significantly improved from 47.0% to 52.1%.
Code (0)
등록된 구현이 없습니다.
Tasks
Action LocalizationDiversityTemporal Action LocalizationWeakly Supervised Action LocalizationWeakly-supervised Temporal Action LocalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploring Denoised Cross-Video Contrast for Weakly-Supervised Temporal Action Localization
Weakly-supervised temporal action localization aims to localize actions in untrimmed videos with only video-level labels. Most existing methods address this problem with a "localization-by-classification" pipeline th…
Action LocalizationContrastive LearningDenoisingPseudo Label+3On Evaluating Weakly Supervised Action Segmentation Methods
Action segmentation is the task of temporally segmenting every frame of an untrimmed video. Weakly supervised approaches to action segmentation, especially from transcripts have been of considerable interest to the compu…
Action SegmentationSegmentationCan MLLMs Guide Weakly-Supervised Temporal Action Localization Tasks?
Recent breakthroughs in Multimodal Large Language Models (MLLMs) have gained significant recognition within the deep learning community, where the fusion of the Video Foundation Models (VFMs) and Large Language Models(LL…
Action LocalizationTemporal Action LocalizationVideo UnderstandingWeakly-supervised Temporal Action LocalizationWeakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature
Weakly-supervised temporal action localization aims to locate action regions and identify action categories in untrimmed videos simultaneously by taking only video-level labels as the supervision. Pseudo label generation…
Action LocalizationPseudo LabelTemporal Action LocalizationWeakly-supervised Temporal Action LocalizationWeakly Supervised Temporal Action Localization via Dual-Prior Collaborative Learning Guided by Multimodal Large Language Models
Recent breakthroughs in Multimodal Large Language Models (MLLMs) have gained significant recognition within the deep learning community, where the fusion of the Video Foundation Models (VFMs) and Large Language Model…
Action LocalizationTemporal Action LocalizationVideo UnderstandingWeakly-supervised Temporal Action Localization+1