Temporal Contrastive Learning with Curriculum
We present ConCur, a contrastive video representation learning method that uses curriculum learning to impose a dynamic sampling strategy in contrastive training. More specifically, ConCur starts the contrastive training with easy positive samples (temporally close and semantically similar clips), and as the training progresses, it increases the temporal span effectively sampling hard positives (temporally away and semantically dissimilar). To learn better context-aware representations, we also propose an auxiliary task of predicting the temporal distance between a positive pair of clips. We conduct extensive experiments on two popular action recognition datasets, UCF101 and HMDB51, on which our proposed method achieves state-of-the-art performance on two benchmark tasks of video action recognition and video retrieval. We explore the impact of encoder backbones and pre-training strategies by using R(2+1)D and C3D encoders and pre-training on Kinetics-400 and Kinetics-200 datasets. Moreover, a detailed ablation study shows the effectiveness of each of the components of our proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionContrastive LearningRepresentation LearningRetrievalTemporal Action LocalizationVideo RetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Weakly-supervised Temporal Path Representation Learning with Contrastive Curriculum Learning -- Extended Version
In step with the digitalization of transportation, we are witnessing a growing range of path-based smart-city applications, e.g., travel-time estimation and travel path ranking. A temporal path(TP) that includes temporal…
Contrastive LearningRepresentation LearningTravel Time EstimationCCML: Curriculum and Contrastive Learning Enhanced Meta-Learner for Personalized Spatial Trajectory Prediction
Spatial trajectory prediction is a fundamental problem for diverse location-based applications. However, existing methods fall short in learning and generalization, and cannot sufficiently capture users’ spatiotemporal p…
Contrastive LearningMeta-LearningTrajectory PredictionConCur: Self-supervised graph representation based on contrastive learning with curriculum negative sampling
Contrastive learning has made breakthrough advancements in graph representation learning, which encourages the representation of positive samples to be close and those of negative samples to be far away. However, existin…
Contrastive LearningGraph Representation LearningNode ClassificationRepresentation Learning+1Efficient Contrastive Learning via Novel Data Augmentation and Curriculum Learning
We introduce EfficientCL, a memory-efficient continual pretraining method that applies contrastive learning with novel data augmentation and curriculum learning. For data augmentation, we stack two types of operation seq…
Continual PretrainingContrastive LearningData AugmentationSentenceACDC: Adaptive Curriculum Planning with Dynamic Contrastive Control for Goal-Conditioned Reinforcement Learning in Robotic Manipulation
Goal-conditioned reinforcement learning has shown considerable potential in robotic manipulation; however, existing approaches remain limited by their reliance on prioritizing collected experience, resulting in suboptima…
Reinforcement LearningContrastive Learning