Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency
Recent works have advanced the performance of self-supervised representation learning by a large margin. The core among these methods is intra-image invariance learning. Two different transformations of one image instance are considered as a positive sample pair, where various tasks are designed to learn invariant representations by comparing the pair. Analogically, for video data, representations of frames from the same video are trained to be closer than frames from other videos, i.e. intra-video invariance. However, cross-video relation has barely been explored for visual representation learning. Unlike intra-video invariance, ground-truth labels of cross-video relation is usually unavailable without human labors. In this paper, we propose a novel contrastive learning method which explores the cross-video relation by using cycle-consistency for general image representation learning. This allows to collect positive sample pairs across different video instances, which we hypothesize will lead to higher-level semantics. We validate our method by transferring our image representation to multiple downstream tasks including visual object tracking, image classification, and action recognition. We show significant improvement over state-of-the-art contrastive learning methods. Project page is available at https://happywu.github.io/cycle_contrast_video.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionContrastive Learningimage-classificationImage ClassificationObject TrackingRelationRepresentation LearningVisual Object TrackingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Cycle-Contrast for Self-Supervised Video Representation Learning
We present Cycle-Contrastive Learning (CCL), a novel self-supervised method for learning video representation. Following a nature that there is a belong and inclusion relation of video and its frames, CCL is designed to …
Action RecognitionContrastive LearningRepresentation LearningRetrieval+1DGGAN: Degradation Guided Generative Adversarial Network for Real-time Endoscopic Video Enhancement
Endoscopic surgery relies on intraoperative video, making image quality a decisive factor for surgical safety and efficacy. Yet, endoscopic videos are often degraded by uneven illumination, tissue scattering, occlusions,…
Contrastive LearningImage EnhancementVideo EnhancementCycleCL: Self-supervised Learning for Periodic Videos
Analyzing periodic video sequences is a key topic in applications such as automatic production systems, remote sensing, medical applications, or physical training. An example is counting repetitions of a physical exercis…
Contrastive LearningSelf-Supervised LearningTripletUnsupervised Contrastive Learning of Image Representations from Ultrasound Videos with Hard Negative Mining
Rich temporal information and variations in viewpoints make video data an attractive choice for learning image representations using unsupervised contrastive learning (UCL) techniques. State-of-the-art (SOTA) contrastive…
Contrastive LearningTowards Contrastive Learning in Music Video Domain
Contrastive learning is a powerful way of learning multimodal representations across various domains such as image-caption retrieval and audio-visual representation learning. In this work, we investigate if these finding…
Contrastive LearningGenre classificationMusic TaggingRepresentation Learning+1