Mining Better Samples for Contrastive Learning of Temporal Correspondence
We present a novel framework for contrastive learning of pixel-level representation using only unlabeled video. Without the need of ground-truth annotation, our method is capable of collecting well-defined positive correspondences by measuring their confidences and well-defined negative ones by appropriately adjusting their hardness during training. This allows us to suppress the adverse impact of ambiguous matches and prevent a trivial solution from being yielded by too hard or too easy negative samples. To accomplish this, we incorporate three different criteria that ranges from a pixel-level matching confidence to a video-level one into a bottom-up pipeline, and plan a curriculum that is aware of current representation power for the adaptive hardness of negative samples during training. With the proposed method, state-of-the-art performance is attained over the latest approaches on several video label propagation tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Weakly Supervised Temporal Sentence Grounding via Positive Sample Mining
The task of weakly supervised temporal sentence grounding (WSTSG) aims to detect temporal intervals corresponding to a language description from untrimmed videos with only video-level video-language correspondence. For a…
Contrastive LearningSentenceTemporal Sentence GroundingGeometric Anchor Correspondence Mining With Uncertainty Modeling for Universal Domain Adaptation
Universal domain adaptation (UniDA) aims to transfer the knowledge learned from a label-rich source domain to a label-scarce target domain without any constraints on the label space. However, domain shift and categor…
Contrastive LearningDomain AdaptationUniversal Domain AdaptationMarginNCE: Robust Sound Localization with a Negative Margin
The goal of this work is to localize sound sources in visual scenes with a self-supervised approach. Contrastive learning in the context of sound source localization leverages the natural correspondence between audio and…
Contrastive LearningSound Source LocalizationGMS-CAVP: Improving Audio-Video Correspondence with Multi-Scale Contrastive and Generative Pretraining
Recent advances in video-audio (V-A) understanding and generation have increasingly relied on joint V-A embeddings, which serve as the foundation for tasks such as cross-modal retrieval and generation. While prior method…
Cross-Modal RetrievalContrastive LearningCLAF: Contrastive Learning with Augmented Features for Imbalanced Semi-Supervised Learning
Due to the advantages of leveraging unlabeled data and learning meaningful representations, semi-supervised learning and contrastive learning have been progressively combined to achieve better performances in popular app…
Contrastive Learningimage-classificationImage Classification