Efficient Human Vision Inspired Action Recognition using Adaptive Spatiotemporal Sampling
Adaptive sampling that exploits the spatiotemporal redundancy in videos is critical for always-on action recognition on wearable devices with limited computing and battery resources. The commonly used fixed sampling strategy is not context-aware and may under-sample the visual content, and thus adversely impacts both computation efficiency and accuracy. Inspired by the concepts of foveal vision and pre-attentive processing from the human visual perception mechanism, we introduce a novel adaptive spatiotemporal sampling scheme for efficient action recognition. Our system pre-scans the global scene context at low-resolution and decides to skip or request high-resolution features at salient regions for further processing. We validate the system on EPIC-KITCHENS and UCF-101 datasets for action recognition, and show that our proposed approach can greatly speed up inference with a tolerable loss of accuracy compared with those from state-of-the-art baselines. Source code is available in https://github.com/knmac/adaptive_spatiotemporal.
Code (1)
Tasks
Action RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Human action recognition with a large-scale brain-inspired photonic computer
The recognition of human actions in video streams is a challenging task in computer vision, with cardinal applications in e.g. brain-computer interface and surveillance. Deep learning has shown remarkable results recentl…
Action RecognitionBrain Computer InterfaceTemporal Action LocalizationMulti-Cue Adaptive Emotion Recognition Network
Expressing and identifying emotions through facial and physical expressions is a significant part of social interaction. Emotion recognition is an essential task in computer vision due to its various applications and mai…
Emotion RecognitionJoint Learning On The Hierarchy Representation for Fine-Grained Human Action Recognition
Fine-grained human action recognition is a core research topic in computer vision. Inspired by the recently proposed hierarchy representation of fine-grained actions in FineGym and SlowFast network for action recognition…
Action RecognitionMulti-Task LearningTemporal Action LocalizationRecognizing Object by Components with Human Prior Knowledge Enhances Adversarial Robustness of Deep Neural Networks
Adversarial attacks can easily fool object recognition systems based on deep neural networks (DNNs). Although many defense methods have been proposed in recent years, most of them can still be adaptively evaded. One reas…
Adversarial RobustnessInductive BiasObjectObject RecognitionHuman Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset
Human Activity Recognition (HAR) primarily relied on traditional RGB cameras to achieve high-performance activity recognition. However, the challenging factors in real-world scenarios, such as insufficient lighting and r…
Activity RecognitionHuman Activity Recognition