Spatio-Temporal Context for Action Detection
Research in action detection has grown in the recentyears, as it plays a key role in video understanding. Modelling the interactions (either spatial or temporal) between actors and their context has proven to be essential for this task. While recent works use spatial features with aggregated temporal information, this work proposes to use non-aggregated temporal information. This is done by adding an attention based method that leverages spatio-temporal interactions between elements in the scene along the clip.The main contribution of this work is the introduction of two cross attention blocks to effectively model the spatial relations and capture short range temporal interactions.Experiments on the AVA dataset show the advantages of the proposed approach that models spatio-temporal relations between relevant elements in the scene, outperforming other methods that model actor interactions with their context by +0.31 mAP.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionVideo UnderstandingSimilar Papers 제목 키워드 기반
Graph-Theoretic Spatiotemporal Context Modeling for Video Saliency Detection
As an important and challenging problem in computer vision, video saliency detection is typically cast as a spatiotemporal context modeling problem over consecutive frames. As a result, a key issue in video saliency dete…
Saliency DetectionVideo Saliency DetectionTACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection
Current state-of-the-art approaches for spatio-temporal action detection have achieved impressive results but remain unsatisfactory for temporal extent detection. The main reason comes from that, there are some ambiguous…
Action DetectionSTEP: Spatio-Temporal Progressive Learning for Video Action Detection
In this paper, we propose Spatio-TEmporal Progressive (STEP) action detector---a progressive learning framework for spatio-temporal action detection in videos. Starting from a handful of coarse-scale proposal cuboids, ou…
Action DetectionAction RecognitionVideo Action DetectionTubeR: Tubelet Transformer for Video Action Detection
We propose TubeR: a simple solution for spatio-temporal video action detection. Different from existing methods that depend on either an off-line actor detector or hand-designed actor-positional hypotheses like proposals…
Action ClassificationAction DetectionAction LocalizationVideo Action Detection+1A Proposal-Based Solution to Spatio-Temporal Action Detection in Untrimmed Videos
Existing approaches for spatio-temporal action detection in videos are limited by the spatial extent and temporal duration of the actions. In this paper, we present a modular system for spatio-temporal action detection i…
Action ClassificationAction DetectionClusteringobject-detection+1