Learning Coupled Spatial-temporal Attention for Skeleton-based Action Recognition
In this paper, we propose a coupled spatial-temporal attention (CSTA) model for skeleton-based action recognition, which aims to figure out the most discriminative joints and frames in spatial and temporal domains simultaneously. Conventional approaches usually consider all the joints or frames in a skeletal sequence equally important, which are unrobust to ambiguous and redundant information. To address this, we first learn two sets of weights for different joints and frames through two subnetworks respectively, which enable the model to have the ability of "paying attention to" the relatively informative section. Then, we calculate the cross product based on the weights of joints and frames for the coupled spatial-temporal attention. Moreover, our CSTA mechanisms can be easily plugged into existing hierarchical CNN models (CSTA-CNN) to realize their function. Extensive experimental results on the recently collected UESTC dataset and the currently largest NTU dataset have shown the effectiveness of our proposed method for skeleton-based action recognition.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionSkeleton Based Action RecognitionSimilar Papers 제목 키워드 기반
Decoupled Spatial-Temporal Attention Network for Skeleton-Based Action Recognition
Dynamic skeletal data, represented as the 2D/3D coordinates of human joints, has been widely studied for human action recognition due to its high-level semantic information and environmental robustness. However, previous…
Action RecognitionSkeleton Based Action RecognitionTemporal Action LocalizationParallel Attention Interaction Network for Few-Shot Skeleton-Based Action Recognition
Learning discriminative features from very few labeled samples to identify novel classes has received increasing attention in skeleton-based action recognition. Existing works aim to learn action-specific embeddings …
Action RecognitionFew-Shot Skeleton-Based Action RecognitionSkeleton Based Action RecognitionModeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks
Recently, skeleton based action recognition gains more popularity due to cost-effective depth sensors coupled with real-time skeleton estimation algorithms. Traditional approaches based on handcrafted features are limite…
3D Action RecognitionAction RecognitionData AugmentationSkeleton Based Action Recognition+1Memory Attention Networks for Skeleton-based Action Recognition
Skeleton-based action recognition task is entangled with complex spatio-temporal variations of skeleton joints, and remains challenging for Recurrent Neural Networks (RNNs). In this work, we propose a temporal-then-spati…
Action RecognitionSkeleton Based Action RecognitionTemporal Action LocalizationSTEP CATFormer: Spatial-Temporal Effective Body-Part Cross Attention Transformer for Skeleton-based Action Recognition
Graph convolutional networks (GCNs) have been widely used and achieved remarkable results in skeleton-based action recognition. We think the key to skeleton-based action recognition is a skeleton hanging in frames, so we…
Action RecognitionSkeleton Based Action Recognition