Self-Supervised Spatiotemporal Feature Learning via Video Rotation Prediction
The success of deep neural networks generally requires a vast amount of training data to be labeled, which is expensive and unfeasible in scale, especially for video collections. To alleviate this problem, in this paper, we propose 3DRotNet: a fully self-supervised approach to learn spatiotemporal features from unlabeled videos. A set of rotations are applied to all videos, and a pretext task is defined as prediction of these rotations. When accomplishing this task, 3DRotNet is actually trained to understand the semantic concepts and motions in videos. In other words, it learns a spatiotemporal video representation, which can be transferred to improve video understanding tasks in small datasets. Our extensive experiments successfully demonstrate the effectiveness of the proposed framework on action recognition, leading to significant improvements over the state-of-the-art self-supervised methods. With the self-supervised pre-trained 3DRotNet from large datasets, the recognition accuracy is boosted up by 20.4% on UCF101 and 16.7% on HMDB51 respectively, compared to the models trained from scratch.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionPredictionSelf-Supervised Action RecognitionTemporal Action LocalizationVideo UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Self-supervised Video Transformer
In this paper, we propose self-supervised training for video transformers using unlabeled video data. From a given video, we create local and global spatiotemporal views with varying spatial sizes and frame rates. Our se…
Action ClassificationAction RecognitionAction Recognition In VideosSelf-Supervised Action Recognition LinearSelf-Supervised Spatiotemporal Learning via Video Clip Order Prediction
We propose a self-supervised spatiotemporal learning technique which leverages the chronological order of videos. Our method can learn the spatiotemporal representation of the video by predicting the order of shuffled cl…
Action RecognitionRetrievalSelf-Supervised Action RecognitionTemporal Action Localization3D-CSL: self-supervised 3D context similarity learning for Near-Duplicate Video Retrieval
In this paper, we introduce 3D-CSL, a compact pipeline for Near-Duplicate Video Retrieval (NDVR), and explore a novel self-supervised learning strategy for video similarity learning. Most previous methods only extract vi…
RetrievalSelf-Supervised LearningTripletVideo Prediction+2Skip-Clip: Self-Supervised Spatiotemporal Representation Learning by Future Clip Order Ranking
Deep neural networks require collecting and annotating large amounts of data to train successfully. In order to alleviate the annotation bottleneck, we propose a novel self-supervised representation learning approach for…
Action RecognitionFuture predictionRepresentation LearningSelf-Supervised Action RecognitionSpatiotemporal Blind-Spot Network with Calibrated Flow Alignment for Self-Supervised Video Denoising
Self-supervised video denoising aims to remove noise from videos without relying on ground truth data, leveraging the video itself to recover clean frames. Existing methods often rely on simplistic feature stacking or ap…
DenoisingOptical Flow EstimationVideo Denoising