Grouped Spatial-Temporal Aggregation for Efficient Action Recognition
Temporal reasoning is an important aspect of video analysis. 3D CNN shows good performance by exploring spatial-temporal features jointly in an unconstrained way, but it also increases the computational cost a lot. Previous works try to reduce the complexity by decoupling the spatial and temporal filters. In this paper, we propose a novel decomposition method that decomposes the feature channels into spatial and temporal groups in parallel. This decomposition can make two groups focus on static and dynamic cues separately. We call this grouped spatial-temporal aggregation (GST). This decomposition is more parameter-efficient and enables us to quantitatively analyze the contributions of spatial and temporal features in different layers. We verify our model on several action recognition tasks that require temporal reasoning and show its effectiveness.
Code (1)
Tasks
Action RecognitionSimilar Papers 제목 키워드 기반
TAN: Temporal Aggregation Network for Dense Multi-label Action Recognition
We present Temporal Aggregation Network (TAN) which decomposes 3D convolutions into spatial and temporal aggregation blocks. By stacking spatial and temporal convolutions repeatedly, TAN forms a deep hierarchical represe…
Action RecognitionTemporal Action LocalizationSkeleton-based Action Recognition via Temporal-Channel Aggregation
Skeleton-based action recognition methods are limited by the semantic extraction of spatio-temporal skeletal maps. However, current methods have difficulty in effectively combining features from both temporal and spatial…
Action RecognitionSkeleton Based Action RecognitionHierarchical Feature Aggregation Networks for Video Action Recognition
Most action recognition methods base on a) a late aggregation of frame level CNN features using average pooling, max pooling, or RNN, among others, or b) spatio-temporal aggregation via 3D convolutions. The first assume …
Action RecognitionTemporal Action LocalizationGC-ConsFlow: Leveraging Optical Flow Residuals and Global Context for Robust Deepfake Detection
The rapid development of Deepfake technology has enabled the generation of highly realistic manipulated videos, posing severe social and ethical challenges. Existing Deepfake detection methods primarily focused on either…
DeepFake DetectionFace SwappingOptical Flow EstimationDistilling Cross-Temporal Contexts for Continuous Sign Language Recognition
Continuous sign language recognition (CSLR) aims to recognize glosses in a sign language video. State-of-the-art methods typically have two modules, a spatial perception module and a temporal aggregation module, whic…
Knowledge DistillationSign Language Recognition