Learning Transferable Self-attentive Representations for Action Recognition in Untrimmed Videos with Weak Supervision
Action recognition in videos has attracted a lot of attention in the past decade. In order to learn robust models, previous methods usually assume videos are trimmed as short sequences and require ground-truth annotations of each video frame/sequence, which is quite costly and time-consuming. In this paper, given only video-level annotations, we propose a novel weakly supervised framework to simultaneously locate action frames as well as recognize actions in untrimmed videos. Our proposed framework consists of two major components. First, for action frame localization, we take advantage of the self-attention mechanism to weight each frame, such that the influence of background frames can be effectively eliminated. Second, considering that there are trimmed videos publicly available and also they contain useful information to leverage, we present an additional module to transfer the knowledge from trimmed videos for improving the classification performance in untrimmed ones. Extensive experiments are conducted on two benchmark datasets (i.e., THUMOS14 and ActivityNet1.3), and experimental results clearly corroborate the efficacy of our method.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionAction Recognition In VideosTemporal Action LocalizationSimilar Papers 제목 키워드 기반
RetaGNN: Relational Temporal Attentive Graph Neural Networks for Holistic Sequential Recommendation
Sequential recommendation (SR) is to accurately recommend a list of items for a user based on her current accessed ones. While new-coming users continuously arrive in the real world, one crucial task is to have inductive…
Sequential RecommendationTARN: Temporal Attentive Relation Network for Few-Shot and Zero-Shot Action Recognition
In this paper we propose a novel Temporal Attentive Relation Network (TARN) for the problems of few-shot and zero-shot action recognition. At the heart of our network is a meta-learning approach that learns to compare re…
Action RecognitionFew-Shot action recognitionFew Shot Action RecognitionMeta-Learning+4Speech-based emotion recognition with self-supervised models using attentive channel-wise correlations and label smoothing
When recognizing emotions from speech, we encounter two common problems: how to optimally capture emotion-relevant information from the speech signal and how to best quantify or categorize the noisy subjective emotion la…
Emotion RecognitionLanguage-based Action Concept Spaces Improve Video Self-Supervised Learning
Recent contrastive language image pre-training has led to learning highly transferable and robust image representations. However, adapting these models to video domains with minimal supervision remains an open problem. W…
Action RecognitionConcept AlignmentSelf-Supervised Action Recognition LinearSelf-Supervised LearningAction Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition
Deep neural networks based purely on attention have been successful across several domains, relying on minimal architectural priors from the designer. In Human Action Recognition (HAR), attention mechanisms have been pri…
Action RecognitionTemporal Action Localization