Action Search: Spotting Actions in Videos and Its Application to Temporal Action Localization
State-of-the-art temporal action detectors inefficiently search the entire video for specific actions. Despite the encouraging progress these methods achieve, it is crucial to design automated approaches that only explore parts of the video which are the most relevant to the actions being searched for. To address this need, we propose the new problem of action spotting in video, which we define as finding a specific action in a video while observing a small portion of that video. Inspired by the observation that humans are extremely efficient and accurate in spotting and finding action instances in video, we propose Action Search, a novel Recurrent Neural Network approach that mimics the way humans spot actions. Moreover, to address the absence of data recording the behavior of human annotators, we put forward the Human Searches dataset, which compiles the search sequences employed by human annotators spotting actions in the AVA and THUMOS14 datasets. We consider temporal action localization as an application of the action spotting problem. Experiments on the THUMOS14 dataset reveal that our model is not only able to explore the video efficiently (observing on average 17.3% of the video) but it also accurately finds human activities with 30.8% mAP.
Code (1)
Tasks
Action LocalizationAction SpottingTemporal Action LocalizationSimilar Papers 제목 키워드 기반
OSL-ActionSpotting: A Unified Library for Action Spotting in Sports Videos
Action spotting is crucial in sports analytics as it enables the precise identification and categorization of pivotal moments in sports matches, providing insights that are essential for performance analysis and tactical…
Action SpottingDecision MakingSports AnalyticsDeep learning for action spotting in association football videos
The task of action spotting consists in both identifying actions and precisely localizing them in time with a single timestamp in long, untrimmed video streams. Automatically extracting those actions is crucial for many …
Action SpottingBenchmarkingDeep LearningSports Analytics+1Towards Active Learning for Action Spotting in Association Football Videos
Association football is a complex and dynamic sport, with numerous actions occurring simultaneously in each game. Analyzing football videos is challenging and requires identifying subtle and diverse spatio-temporal patte…
Action SpottingActive LearningActionSpotter: Deep Reinforcement Learning Framework for Temporal Action Spotting in Videos
Summarizing video content is an important task in many applications. This task can be defined as the computation of the ordered list of actions present in a video. Such a list could be extracted using action detection al…
Action DetectionAction SpottingDeep Reinforcement Learningreinforcement-learning+2A Context-Aware Loss Function for Action Spotting in Soccer Videos
In video understanding, action spotting consists in temporally localizing human-induced events annotated with single timestamps. In this paper, we propose a novel loss function that specifically considers the temporal co…
Action SpottingVideo Understanding