Joint Image-Instance Spatial-Temporal Attention for Few-shot Action Recognition
Few-shot Action Recognition (FSAR) constitutes a crucial challenge in computer vision, entailing the recognition of actions from a limited set of examples. Recent approaches mainly focus on employing image-level features to construct temporal dependencies and generate prototypes for each action category. However, a considerable number of these methods utilize mainly image-level features that incorporate background noise and focus insufficiently on real foreground (action-related instances), thereby compromising the recognition capability, particularly in the few-shot scenario. To tackle this issue, we propose a novel joint Image-Instance level Spatial-temporal attention approach (I2ST) for Few-shot Action Recognition. The core concept of I2ST is to perceive the action-related instances and integrate them with image features via spatial-temporal attention. Specifically, I2ST consists of two key components: Action-related Instance Perception and Joint Image-Instance Spatial-temporal Attention. Given the basic representations from the feature extractor, the Action-related Instance Perception is introduced to perceive action-related instances under the guidance of a text-guided segmentation model. Subsequently, the Joint Image-Instance Spatial-temporal Attention is used to construct the feature dependency between instances and images...
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionFew-Shot action recognitionFew Shot Action RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Towards Robust Video Instance Segmentation with Temporal-Aware Transformer
Most existing transformer based video instance segmentation methods extract per frame features independently, hence it is challenging to solve the appearance deformation problem. In this paper, we observe the temporal in…
DecoderInstance SegmentationSemantic SegmentationVideo Instance SegmentationInstAP: Instance-Aware Vision-Language Pre-Train for Spatial-Temporal Understanding
Current vision-language pre-training (VLP) paradigms excel at global scene understanding but struggle with instance-level reasoning due to global-only supervision. We introduce InstAP, an Instance-Aware Pre-training fram…
Scene UnderstandingEVA02-AT: Egocentric Video-Language Understanding with Spatial-Temporal Rotary Positional Embeddings and Symmetric Optimization
Egocentric video-language understanding demands both high efficiency and accurate spatial-temporal modeling. Existing approaches face three key challenges: 1) Excessive pre-training cost arising from multi-stage pre-trai…
Multi-Instance RetrievalRetrievalVideo UnderstandingA Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection
Remote sensing image change detection (CD) is done to identify desired significant changes between bitemporal images. Given two co-registered images taken at different times, the illumination variations and misregistrati…
Change DetectionSpatial Hierarchy and Temporal Attention Guided Cross Masking for Self-supervised Skeleton-based Action Recognition
In self-supervised skeleton-based action recognition, the mask reconstruction paradigm is gaining interest in enhancing model refinement and robustness through effective masking. However, previous works primarily relied …
3D Action RecognitionAction RecognitionSelf-supervised Skeleton-based Action RecognitionSkeleton Based Action Recognition