Temporal Action Detection with Structured Segment Networks
Detecting actions in untrimmed videos is an important yet challenging task. In this paper, we present the structured segment network (SSN), a novel framework which models the temporal structure of each action instance via a structured temporal pyramid. On top of the pyramid, we further introduce a decomposed discriminative model comprising two classifiers, respectively for classifying actions and determining completeness. This allows the framework to effectively distinguish positive proposals from background or incomplete ones, thus leading to both accurate recognition and localization. These components are integrated into a unified network that can be efficiently trained in an end-to-end fashion. Additionally, a simple yet effective temporal action proposal scheme, dubbed temporal actionness grouping (TAG) is devised to generate high quality action proposals. On two challenging benchmarks, THUMOS14 and ActivityNet, our method remarkably outperforms previous state-of-the-art methods, demonstrating superior accuracy and strong adaptivity in handling actions with various temporal structures.
Code (6)
Tasks
Action DetectionAction RecognitionTAGSimilar Papers 제목 키워드 기반
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action
Understanding causal event relationships and achieving fine-grained temporal grounding in videos remain challenging for vision-language models. Existing methods either compress video tokens to reduce temporal resolution,…
Dense CaptioningHighlight DetectionVideo SegmentationVideo Semantic Segmentation+1Unsupervised Human Action Detection by Action Matching
We propose a new task of unsupervised action detection by action matching. Given two long videos, the objective is to temporally detect all pairs of matching video segments. A pair of video segments are matched if they s…
Action DetectionActivity RecognitionSegTAD: Precise Temporal Action Detection via Semantic Segmentation
Temporal action detection (TAD) is an important yet challenging task in video analysis. Most existing works draw inspiration from image object detection and tend to reformulate it as a proposal generation - classificatio…
Action Detectionobject-detectionObject DetectionSegmentation+1Exploring Vision-Language Models for Open-Vocabulary Zero-Shot Action Segmentation
Temporal Action Segmentation (TAS) requires dividing videos into action segments, yet the vast space of activities and alternative breakdowns makes collecting comprehensive datasets infeasible. Existing methods remain li…
Action SegmentationTemporal Action Detection Using a Statistical Language Model
While current approaches to action recognition on pre-segmented video clips already achieve high accuracies, temporal action detection is still far from comparably good results. Automatically locating and classifying the…
Action DetectionAction RecognitionLanguage ModelingLanguage Modelling+2