CAG-QIL: Context-Aware Actionness Grouping via Q Imitation Learning for Online Temporal Action Localization
Temporal action localization has been one of the most popular tasks in video understanding, due to the importance of detecting action instances in videos. However, not much progress has been made on extending it to work in an online fashion, although many video related tasks can benefit by going online with the growing video streaming services. To this end, we introduce a new task called Online Temporal Action Localization (On-TAL), in which the goal is to immediately detect action instances from an untrimmed streaming video. The online setting makes the new task very challenging as the actionness decision for every frame has to be made without access to future frames and also because post-processing methods cannot be used to modify past action proposals. We propose a novel framework, Context-Aware Actionness Grouping (CAG) as a solution for On-TAL and train it with the imitation learning algorithm, which allows us to avoid sophisticated reward engineering. Evaluation of our work on THUMOS14 and Activitynet1.3 shows significant improvement over non-naive baselines, demonstrating the effectiveness of our approach. As a by-product, our method can also be used for the Online Detection of Action Start (ODAS), in which our method also outperforms previous state-of-the-art models.
Code (0)
등록된 구현이 없습니다.
Tasks
Action LocalizationImitation LearningTemporal Action LocalizationVideo UnderstandingSimilar Papers 제목 키워드 기반
CTAP: Complementary Temporal Action Proposal Generation
Temporal action proposal generation is an important task, akin to object proposals, temporal action proposals are intended to capture "clips" or temporal intervals in videos that are likely to contain an action. Previous…
Temporal Action Proposal GenerationActionness Estimation Using Hybrid Fully Convolutional Networks
Actionness was introduced to quantify the likelihood of containing a generic action instance at a specific location. Accurate and efficient estimation of actionness is important in video analysis and may benefit other re…
Action DetectionAction RecognitionTemporal Action LocalizationAccurate Temporal Action Proposal Generation with Relation-Aware Pyramid Network
Accurate temporal action proposals play an important role in detecting actions from untrimmed videos. The existing approaches have difficulties in capturing global contextual information and simultaneously localizing act…
RelationTemporal Action Proposal GenerationActionness Inconsistency-guided Contrastive Learning for Weakly-supervised Temporal Action Localization
Weakly-supervised temporal action localization (WTAL) aims to detect action instances given only video-level labels. To address the challenge, recent methods commonly employ a two-branch framework, consisting of a class-…
Action LocalizationContrastive LearningRepresentation LearningTemporal Action Localization+2Searching Action Proposals via Spatial Actionness Estimation and Temporal Path Inference and Tracking
In this paper, we address the problem of searching action proposals in unconstrained video clips. Our approach starts from actionness estimation on frame-level bounding boxes, and then aggregates the bounding boxes belon…