Learning clip representations for skeleton-based 3d action recognition
This paper presents a new representation of skeleton sequences for 3D action recognition. Existing methods based on hand-crafted features or recurrent neural networks cannot adequately capture the complex spatial structures and the long-term temporal dynamics of the skeleton sequences, which are very important to recognize the actions. In this paper, we propose to transform each channel of the 3D coordinates of a skeleton sequence into a clip. Each frame of the generated clip represents the temporal information of the entire skeleton sequence and one particular spatial relationship between the skeleton joints. The entire clip incorporates multiple frames with different spatial relationships, which provide useful spatial structural information of the human skeleton. We also propose a multitask convolutional neural network (MTCNN) to learn the generated clips for action recognition. The proposed MTCNN processes all the frames of the generated clips in parallel to explore the spatial and temporal information of the skeleton sequences. The proposed method has been extensively tested on six challenging benchmark datasets. Experimental results consistently demonstrate the superiority of the proposed clip representation and the feature learning method for 3D action recognition compared to the existing techniques.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Action RecognitionAction RecognitionSkeleton Based Action RecognitionSimilar Papers 제목 키워드 기반
SkelHCC: A Hyperbolic CLIP-Driven Cache Adaptation Framework for Skeleton-based One-Shot Action Recognition
Skeleton-based action recognition aims to understand human behaviors from body joint sequences and is especially challenging in the one-shot setting, where only a single labeled exemplar is available for each novel actio…
Action RecognitionA New Representation of Skeleton Sequences for 3D Action Recognition
This paper presents a new method for 3D action recognition with skeleton sequences (i.e., 3D trajectories of human skeleton joints). The proposed method first transforms each skeleton sequence into three clips each consi…
3D Action RecognitionAction RecognitionMulti-Task LearningSkeleton Based Action Recognition+1Skeleton-Based Action Recognition with Spatial Reasoning and Temporal Stack Learning
Skeleton-based action recognition has made great progress recently, but many problems still remain unsolved. For example, most of the previous methods model the representations of skeleton sequences without abundant spat…
Action RecognitionGraph Neural NetworkHuman-Object Interaction DetectionSkeleton Based Action Recognition+2T-MOR: Learning Motion-Aware Skeleton Representations for Human Action Recognition
Vision-language models such as CLIP have recently achieved strong performance on a wide range of visual understanding tasks. However, most existing models rely primarily on appearance-level supervision from images or vid…
Action ClassificationContrastive LearningAction UnderstandingAction RecognitionSelf-Supervised 3D Skeleton Action Representation Learning With Motion Consistency and Continuity
Recently, self-supervised learning (SSL) has been proved very effective and it can help boost the performance in learning representations from unlabeled data in the image domain. Yet, very little is explored about it…
Action RecognitionRepresentation LearningSelf-Supervised LearningSkeleton Based Action Recognition