Temporal-Spatial Mapping for Action Recognition
Deep learning models have enjoyed great success for image related computer vision tasks like image classification and object detection. For video related tasks like human action recognition, however, the advancements are not as significant yet. The main challenge is the lack of effective and efficient models in modeling the rich temporal spatial information in a video. We introduce a simple yet effective operation, termed Temporal-Spatial Mapping (TSM), for capturing the temporal evolution of the frames by jointly analyzing all the frames of a video. We propose a video level 2D feature representation by transforming the convolutional features of all frames to a 2D feature map, referred to as VideoMap. With each row being the vectorized feature representation of a frame, the temporal-spatial features are compactly represented, while the temporal dynamic evolution is also well embedded. Based on the VideoMap representation, we further propose a temporal attention model within a shallow convolutional neural network to efficiently exploit the temporal-spatial dynamics. The experiment results show that the proposed scheme achieves the state-of-the-art performance, with 4.2% accuracy gain over Temporal Segment Network (TSN), a competing baseline method, on the challenging human action benchmark dataset HMDB51.
Code (0)
등록된 구현이 없습니다.
Tasks
Action Recognitionimage-classificationImage Classificationobject-detectionObject DetectionTemporal Action LocalizationSimilar Papers 제목 키워드 기반
Neuron: Learning Context-Aware Evolving Representations for Zero-Shot Skeleton Action Recognition
Zero-shot skeleton action recognition is a non-trivial task that requires robust unseen generalization with prior knowledge from only seen classes and shared semantics. Existing methods typically build the skeleton-seman…
Action RecognitionHuman-AI Divergence in Ego-centric Action Recognition under Spatial and Spatiotemporal Manipulations
Humans consistently outperform state-of-the-art AI models in action recognition, particularly in challenging real-world conditions involving low resolution, occlusion, and visual clutter. Understanding the sources of thi…
Action RecognitionDynamic Spatial-temporal Hypergraph Convolutional Network for Skeleton-based Action Recognition
Skeleton-based action recognition relies on the extraction of spatial-temporal topological information. Hypergraphs can establish prior unnatural dependencies for the skeleton. However, the existing methods only focus on…
Action RecognitionSkeleton Based Action RecognitionR-STAN: Residual Spatial-Temporal Attention Network for Action Recognition
Two-stream network architecture has the ability to capture temporal and spatial features from videos simultaneously and has achieved excellent performance on video action recognition tasks. However, there is a fair amoun…
Action RecognitionTemporal Action LocalizationOne-Shot Action Recognition via Multi-Scale Spatial-Temporal Skeleton Matching
One-shot skeleton action recognition, which aims to learn a skeleton action recognition model with a single training sample, has attracted increasing interest due to the challenge of collecting and annotating large-scale…
Action Recognition