Skeleton-Contrastive 3D Action Representation Learning
This paper strives for self-supervised learning of a feature space suitable for skeleton-based action recognition. Our proposal is built upon learning invariances to input skeleton representations and various skeleton augmentations via a noise contrastive estimation. In particular, we propose inter-skeleton contrastive learning, which learns from multiple different input skeleton representations in a cross-contrastive manner. In addition, we contribute several skeleton-specific spatial and temporal augmentations which further encourage the model to learn the spatio-temporal dynamics of skeleton data. By learning similarities between different skeleton representations as well as augmented views of the same sequence, the network is encouraged to learn higher-level semantics of the skeleton data than when only using the augmented views. Our approach achieves state-of-the-art performance for self-supervised learning from skeleton data on the challenging PKU and NTU datasets with multiple downstream tasks, including action recognition, action retrieval and semi-supervised learning. Code is available at https://github.com/fmthoker/skeleton-contrast.
Code (1)
Tasks
Action RecognitionContrastive LearningFew-Shot Skeleton-Based Action RecognitionRepresentation LearningRetrievalSelf-Supervised LearningSelf-supervised Skeleton-based Action RecognitionSkeleton Based Action RecognitionSimilar Papers 제목 키워드 기반
Part Aware Contrastive Learning for Self-Supervised Action Recognition
In recent years, remarkable results have been achieved in self-supervised action recognition using skeleton sequences with contrastive learning. It has been observed that the semantic distinction of human action features…
Action RecognitionContrastive LearningData AugmentationRepresentation Learning+3MS-CLR: Multi-Skeleton Contrastive Learning for Human Action Recognition
Contrastive learning has gained significant attention in skeleton-based action recognition for its ability to learn robust representations from unlabeled data. However, existing methods rely on a single skeleton conventi…
Contrastive LearningAction RecognitionSkeleton-Snippet Contrastive Learning with Multiscale Feature Fusion for Action Localization
The self-supervised pretraining paradigm has achieved great success in learning 3D action representations for skeleton-based action recognition using contrastive learning. However, learning effective representations for …
Temporal Action LocalizationContrastive LearningAction RecognitionTransfer LearningContrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognition
Self-supervised skeleton-based action recognition with contrastive learning has attracted much attention. Recent literature shows that data augmentation and large sets of contrastive pairs are crucial in learning such re…
Action RecognitionContrastive LearningData AugmentationSelf-supervised Skeleton-based Action Recognition+1Variational Contrastive Learning for Skeleton-based Action Recognition
In recent years, self-supervised representation learning for skeleton-based action recognition has advanced with the development of contrastive learning methods. However, most of contrastive paradigms are inherently disc…
Self-Supervised LearningRepresentation LearningContrastive LearningAction Recognition