Self-Supervised Visual Learning by Variable Playback Speeds Prediction of a Video
We propose a self-supervised visual learning method by predicting the variable playback speeds of a video. Without semantic labels, we learn the spatio-temporal visual representation of the video by leveraging the variations in the visual appearance according to different playback speeds under the assumption of temporal coherence. To learn the spatio-temporal visual variations in the entire video, we have not only predicted a single playback speed but also generated clips of various playback speeds and directions with randomized starting points. Hence the visual representation can be successfully learned from the meta information (playback speeds and directions) of the video. We also propose a new layer dependable temporal group normalization method that can be applied to 3D convolutional networks to improve the representation learning performance where we divide the temporal features into several groups and normalize each one using the different corresponding parameters. We validate the effectiveness of our method by fine-tuning it to the action recognition and video retrieval tasks on UCF-101 and HMDB-51.
Code (1)
Tasks
Action RecognitionRepresentation LearningRetrievalSelf-Supervised Action RecognitionSelf-Supervised LearningVideo RetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variable-Speed Teaching-Playback as Real-World Data Augmentation for Imitation Learning
Because imitation learning relies on human demonstrations in hard-to-simulate settings, the inclusion of force control in this method has resulted in a shortage of training data, even with a simple change in speed. Altho…
Data AugmentationImitation LearningPositionRobot ManipulationASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistency
We study self-supervised video representation learning, which is a challenging task due to 1) lack of labels for explicit supervision; 2) unstructured and noisy visual information. Existing methods mainly use contrastive…
Action RecognitionRepresentation LearningRetrievalVideo RetrievalSpeed Co-Augmentation for Unsupervised Audio-Visual Pre-training
This work aims to improve unsupervised audio-visual pre-training. Inspired by the efficacy of data augmentation in visual contrastive learning, we propose a novel speed co-augmentation method that randomly changes the pl…
Contrastive LearningData AugmentationDiversitySeeing Fast and Slow: Learning the Flow of Time in Videos
How can we tell whether a video has been sped up or slowed down? How can we generate videos at different speeds? Although videos have been central to modern computer vision research, little attention has been paid to per…
Video GenerationAIx Speed: Playback Speed Optimization Using Listening Comprehension of Speech Recognition Models
Since humans can listen to audio and watch videos at faster speeds than actually observed, we often listen to or watch these pieces of content at higher playback speeds to increase the time efficiency of content comprehe…
speech-recognitionSpeech Recognition