Self-Supervised Video Representation Learning With Odd-One-Out Networks
We propose a new self-supervised CNN pre-training technique based on a novel auxiliary task called "odd-one-out learning". In this task, the machine is asked to identify the unrelated or odd element from a set of otherwise related elements. We apply this technique to self-supervised video representation learning where we sample subsequences from videos and ask the network to learn to predict the odd video subsequence. The odd video subsequence is sampled such that it has wrong temporal order of frames while the even ones have the correct temporal order. Therefore, to generate a odd-one-out question no manual annotation is required. Our learning machine is implemented as multi-stream convolutional neural network, which is learned end-to-end. Using odd-one-out networks, we learn temporal representations for videos that generalizes to other related tasks such as action recognition. On action classification, our method obtains 60.3\% on the UCF101 dataset using only UCF101 data for training which is approximately 10% better than current state-of-the-art self-supervised learning methods. Similarly, on HMDB51 dataset we outperform self-supervised state-of-the art methods by 12.7% on action classification task.
Code (0)
등록된 구현이 없습니다.
Tasks
Action ClassificationAction RecognitionGeneral ClassificationOdd One OutRepresentation LearningSelf-Supervised Action RecognitionSelf-Supervised LearningTemporal Action LocalizationSimilar Papers 제목 키워드 기반
ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpoints
Learning self-supervised video representation predominantly focuses on discriminating instances generated from simple data augmentation schemes. However, the learned representation often fails to generalize over unseen c…
Data AugmentationAdvancing Video Self-Supervised Learning via Image Foundation Models
In the past decade, image foundation models (IFMs) have achieved unprecedented progress. However, the potential of directly using IFMs for video self-supervised representation learning has largely been overlooked. In thi…
GPURepresentation LearningSelf-Supervised LearningVideo Cloze Procedure for Self-Supervised Spatio-Temporal Learning
We propose a novel self-supervised method, referred to as Video Cloze Procedure (VCP), to learn rich spatial-temporal representations. VCP first generates "blanks" by withholding video clips and then creates "options" by…
Action RecognitionRepresentation LearningRetrievalSelf-Supervised Action Recognition+3Self-supervised and Weakly Supervised Contrastive Learning for Frame-wise Action Representations
Previous work on action representation learning focused on global representations for short video clips. In contrast, many practical applications, such as video alignment, strongly demand learning the intensive represent…
Action ClassificationContrastive LearningRepresentation LearningRetrieval+2Self-Supervised Representation Learning for Visual Anomaly Detection
Self-supervised learning allows for better utilization of unlabelled data. The feature representation obtained by self-supervision can be used in downstream tasks such as classification, object detection, segmentation, a…
Anomaly DetectionGeneral Classificationobject-detectionObject Detection+4