Rethinking Learning Approaches for Long-Term Action Anticipation
Action anticipation involves predicting future actions having observed the initial portion of a video. Typically, the observed video is processed as a whole to obtain a video-level representation of the ongoing activity in the video, which is then used for future prediction. We introduce ANTICIPATR which performs long-term action anticipation leveraging segment-level representations learned using individual segments from different activities, in addition to a video-level representation. We propose a two-stage learning approach to train a novel transformer-based model that uses these two types of representations to directly predict a set of future action instances over any given anticipation duration. Results on Breakfast, 50Salads, Epic-Kitchens-55, and EGTEA Gaze+ datasets demonstrate the effectiveness of our approach.
Code (1)
Tasks
Action AnticipationFuture predictionLong Term Action AnticipationSimilar Papers 제목 키워드 기반
Rethinking Anticipation Tasks: Uncertainty-aware Anticipation of Sparse Surgical Instrument Usage for Context-aware Assistance
Intra-operative anticipation of instrument usage is a necessary component for context-aware assistance in surgery, e.g. for instrument preparation or semi-automation of robotic tasks. However, the sparsity of instrument …
Rethinking Video Human-Object Interaction: Set Prediction over Time for Unified Detection and Anticipation
Video-based human-object interaction (HOI) understanding requires both detecting ongoing interactions and anticipating their future evolution. However, existing methods usually treat anticipation as a downstream forecast…
Representation LearningObject LocalizationIntention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipation
Long-term action anticipation from egocentric video is critical for applications such as human-computer interaction and assistive technologies, where anticipating user intent enables proactive and context-aware AI assist…
Reinforcement LearningAction AnticipationFuture Transformer for Long-term Action Anticipation
The task of predicting future actions from a video is crucial for a real-world agent interacting with others. When anticipating actions in the distant future, we humans typically consider long-term relations over the who…
Action AnticipationLong Term Action AnticipationLong Term AnticipationTime-Conditioned Action Anticipation in One Shot
The goal of human action anticipation is to predict future actions. Ideally, in real-world applications such as video surveillance and self-driving systems, future actions should not only be predicted with high accuracy …
Action AnticipationLong Term Action Anticipation