Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition
Semi-supervised action recognition aims to improve spatio-temporal reasoning ability with a few labeled data in conjunction with a large amount of unlabeled data. Albeit recent advancements, existing powerful methods are still prone to making ambiguous predictions under scarce labeled data, embodied as the limitation of distinguishing different actions with similar spatio-temporal information. In this paper, we approach this problem by empowering the model two aspects of capability, namely discriminative spatial modeling and temporal structure modeling for learning discriminative spatio-temporal representations. Specifically, we propose an Adaptive Contrastive Learning~(ACL) strategy. It assesses the confidence of all unlabeled samples by the class prototypes of the labeled data, and adaptively selects positive-negative samples from a pseudo-labeled sample bank to construct contrastive learning. Additionally, we introduce a Multi-scale Temporal Learning~(MTL) strategy. It could highlight informative semantics from long-term clips and integrate them into the short-term clip while suppressing noisy information. Afterwards, both of these two new techniques are integrated in a unified framework to encourage the model to make accurate predictions. Extensive experiments on UCF101, HMDB51 and Kinetics400 show the superiority of our method over prior state-of-the-art approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionContrastive LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Spatiotemporal Decouple-and-Squeeze Contrastive Learning for Semi-Supervised Skeleton-based Action Recognition
Contrastive learning has been successfully leveraged to learn action representations for addressing the problem of semi-supervised skeleton-based action recognition. However, most contrastive learning-based methods only …
Action RecognitionContrastive LearningSelf-Supervised Human Action RecognitionSkeleton Based Action RecognitionBuilding Autocorrelation-Aware Representations for Fine-Scale Spatiotemporal Prediction
Many scientific prediction problems have spatiotemporal data- and modeling-related challenges in handling complex variations in space and time using only sparse and unevenly distributed observations. This paper presents …
feature selectionTime SeriesTime Series AnalysisSpatiotemporal Classification with limited labels using Constrained Clustering for large datasets
Creating separable representations via representation learning and clustering is critical in analyzing large unstructured datasets with only a few labels. Separable representations can lead to supervised models with bett…
ClusteringConstrained ClusteringRepresentation LearningBrainSTR: Spatio-Temporal Contrastive Learning for Interpretable Dynamic Brain Network Modeling
Dynamic functional connectivity captures time-varying brain states for better neuropsychiatric diagnosis and spatio-temporal interpretability, i.e., identifying when discriminative disease signatures emerge and where the…
Contrastive LearningClassification of sparsely labeled spatio-temporal data through semi-supervised adversarial learning
In recent years, Generative Adversarial Networks (GAN) have emerged as a powerful method for learning the mapping from noisy latent spaces to realistic data samples in high-dimensional space. So far, the development and …
General ClassificationRetrieval