Random Temporal Skipping for Multirate Video Analysis
Current state-of-the-art approaches to video understanding adopt temporal jittering to simulate analyzing the video at varying frame rates. However, this does not work well for multirate videos, in which actions or subactions occur at different speeds. The frame sampling rate should vary in accordance with the different motion speeds. In this work, we propose a simple yet effective strategy, termed random temporal skipping, to address this situation. This strategy effectively handles multirate videos by randomizing the sampling rate during training. It is an exhaustive approach, which can potentially cover all motion speed variations. Furthermore, due to the large temporal skipping, our network can see video clips that originally cover over 100 frames. Such a time range is enough to analyze most actions/events. We also introduce an occlusion-aware optical flow learning method that generates improved motion maps for human action recognition. Our framework is end-to-end trainable, runs in real-time, and achieves state-of-the-art performance on six widely adopted video benchmarks.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionOptical Flow EstimationTemporal Action LocalizationVideo UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bidirectional Multirate Reconstruction for Temporal Modeling in Videos
Despite the recent success of neural networks in image feature learning, a major problem in the video domain is the lack of sufficient labeled data for learning to model temporal information. In this paper, we propose an…
Event DetectionVideo CaptioningVideo Representation Learning by Recognizing Temporal Transformations
We introduce a novel self-supervised learning approach to learn representations of videos that are responsive to changes in the motion dynamics. Our representations can be learned from data without human annotation and p…
Action RecognitionRepresentation LearningSelf-Supervised LearningPerformance Analysis of Multirate Systems: A Direct Frequency-Domain Identification Approach
Frequency-domain performance analysis of intersample behavior in sampled-data and multirate systems is challenging due to the lack of a frequency-separation principle, and systematic identification techniques are lacking…
Matrix Pencil-Based Analysis of Multirate Simulation Schemes
This paper focuses on multirate time-domain simulations of power system models. It proposes a matrix pencil-based approach to evaluate the spurious numerical deformation introduced into power system dynamics by a given m…
Learning Video Representations by Transforming Time
We introduce a novel self-supervised learning approach to learn representations of videos that are responsive to changes in the motion dynamics. Our representations can be learned from data without human annotation and p…
Action RecognitionSelf-Supervised Learning