Self-Supervised Video Representation Learning with Motion-Contrastive Perception
Visual-only self-supervised learning has achieved significant improvement in video representation learning. Existing related methods encourage models to learn video representations by utilizing contrastive learning or designing specific pretext tasks. However, some models are likely to focus on the background, which is unimportant for learning video representations. To alleviate this problem, we propose a new view called long-range residual frame to obtain more motion-specific information. Based on this, we propose the Motion-Contrastive Perception Network (MCPNet), which consists of two branches, namely, Motion Information Perception (MIP) and Contrastive Instance Perception (CIP), to learn generic video representations by focusing on the changing areas in videos. Specifically, the MIP branch aims to learn fine-grained motion features, and the CIP branch performs contrastive learning to learn overall semantics information for each instance. Experiments on two benchmark datasets UCF-101 and HMDB-51 show that our method outperforms current state-of-the-art visual-only self-supervised approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningRepresentation LearningSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MaCLR: Motion-aware Contrastive Learning of Representations for Videos
We present MaCLR, a novel method to explicitly perform cross-modal self-supervised video representations learning from visual and motion modalities. Compared to previous video representation learning methods that mostly …
Action DetectionAction RecognitionContrastive LearningRepresentation LearningSCVRL: Shuffled Contrastive Video Representation Learning
We propose SCVRL, a novel contrastive-based framework for self-supervised learning for videos. Differently from previous contrast learning based methods that mostly focus on learning visual semantics (e.g., CVRL), SCVRL …
Contrastive LearningRepresentation LearningSelf-Supervised LearningMotion-Focused Contrastive Learning of Video Representations
Motion, as the most distinct phenomenon in a video to involve the changes over time, has been unique and critical to the development of video representation learning. In this paper, we ask the question: how important is …
Contrastive LearningData AugmentationOptical Flow EstimationRepresentation LearningMotion Sensitive Contrastive Learning for Self-supervised Video Representation
Contrastive learning has shown great potential in video representation learning. However, existing approaches fail to sufficiently exploit short-term motion dynamics, which are crucial to various down-stream video unders…
Contrastive LearningRepresentation LearningRetrievalVideo Classification+2Hierarchical Contrastive Motion Learning for Video Action Recognition
One central question for video action recognition is how to model motion. In this paper, we present hierarchical contrastive motion learning, a new self-supervised learning framework to extract effective motion represent…
Action RecognitionContrastive LearningSelf-Supervised LearningTemporal Action Localization