Collaboratively Self-supervised Video Representation Learning for Action Recognition
Considering the close connection between action recognition and human pose estimation, we design a Collaboratively Self-supervised Video Representation (CSVR) learning framework specific to action recognition by jointly factoring in generative pose prediction and discriminative context matching as pretext tasks. Specifically, our CSVR consists of three branches: a generative pose prediction branch, a discriminative context matching branch, and a video generating branch. Among them, the first one encodes dynamic motion feature by utilizing Conditional-GAN to predict the human poses of future frames, and the second branch extracts static context features by contrasting positive and negative video feature and I-frame feature pairs. The third branch is designed to generate both current and future video frames, for the purpose of collaboratively improving dynamic motion features and static context features. Extensive experiments demonstrate that our method achieves state-of-the-art performance on multiple popular video datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionPose EstimationPose PredictionRepresentation LearningSimilar Papers 제목 키워드 기반
Self-supervised and Weakly Supervised Contrastive Learning for Frame-wise Action Representations
Previous work on action representation learning focused on global representations for short video clips. In contrast, many practical applications, such as video alignment, strongly demand learning the intensive represent…
Action ClassificationContrastive LearningRepresentation LearningRetrieval+2Exploring Relations in Untrimmed Videos for Self-Supervised Learning
Existing video self-supervised learning methods mainly rely on trimmed videos for model training. However, trimmed datasets are manually annotated from untrimmed videos. In this sense, these methods are not really self-s…
Action RecognitionChange DetectionRetrievalSelf-Supervised Learning+1Self-Supervised Video Representation Learning With Odd-One-Out Networks
We propose a new self-supervised CNN pre-training technique based on a novel auxiliary task called "odd-one-out learning". In this task, the machine is asked to identify the unrelated or odd element from a set of otherwi…
Action ClassificationAction RecognitionGeneral ClassificationOdd One Out+4Cross-Attentional Audio-Visual Fusion for Weakly-Supervised Action Localization
Temporally localizing actions in videos is one of the key components for video understanding. Learning from weakly-labelled data is seen a potential solution towards avoiding expensive frame-level annotations. Different …
Action LocalizationVideo UnderstandingWeakly Supervised Action LocalizationSelf-supervised Spatiotemporal Representation Learning by Exploiting Video Continuity
Recent self-supervised video representation learning methods have found significant success by exploring essential properties of videos, e.g. speed, temporal order, etc. This work exploits an essential yet under-explored…
Action LocalizationAction RecognitionRepresentation LearningRetrieval+1