Compressed Video Action Recognition
Training robust deep video representations has proven to be much more challenging than learning deep image representations. This is in part due to the enormous size of raw video streams and the high temporal redundancy; the true and interesting signal is often drowned in too much irrelevant data. Motivated by that the superfluous information can be reduced by up to two orders of magnitude by video compression (using H.264, HEVC, etc.), we propose to train a deep network directly on the compressed video. This representation has a higher information density, and we found the training to be easier. In addition, the signals in a compressed video provide free, albeit noisy, motion information. We propose novel techniques to use them effectively. Our approach is about 4.6 times faster than Res3D and 2.7 times faster than ResNet-152. On the task of action recognition, our approach outperforms all the other methods on the UCF-101, HMDB-51, and Charades dataset.
Code (1)
Tasks
Action ClassificationAction RecognitionTemporal Action LocalizationVideo CompressionSimilar Papers 제목 키워드 기반
Faster and Accurate Compressed Video Action Recognition Straight from the Frequency Domain
Human action recognition has become one of the most active field of research in computer vision due to its wide range of applications, like surveillance, medical, industrial environments, smart homes, among others. Recen…
Action RecognitionTemporal Action LocalizationFlow-Distilled IP Two-Stream Networks for Compressed Video Action Recognition
Two-stream networks have achieved great success in video recognition. A two-stream network combines a spatial stream of RGB frames and a temporal stream of Optical Flow to make predictions. However, the temporal redundan…
Action RecognitionOptical Flow EstimationTemporal Action LocalizationVideo Recognition+1Speeding Up Action Recognition Using Dynamic Accumulation of Residuals in Compressed Domain
With the widespread use of installed cameras, video-based monitoring approaches have seized considerable attention for different purposes like assisted living. Temporal redundancy and the sheer size of raw videos are the…
Action RecognitionVideo UnderstandingMimic The Raw Domain: Accelerating Action Recognition in the Compressed Domain
Video understanding usually requires expensive computation that prohibits its deployment, yet videos contain significant spatiotemporal redundancy that can be exploited. In particular, operating directly on the motion ve…
Action RecognitionVideo RecognitionVideo UnderstandingMM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognition
This paper presents a pure transformer-based approach, dubbed the Multi-Modal Video Transformer (MM-ViT), for video action recognition. Different from other schemes which solely utilize the decoded RGB frames, MM-ViT ope…
Action RecognitionOptical Flow EstimationTemporal Action Localization