Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videos
This work focuses on anticipating long-term human actions, particularly using short video segments, which can speed up editing workflows through improved suggestions while fostering creativity by suggesting narratives. To this end, we imbue a transformer network with a symbolic knowledge graph for action anticipation in video segments by boosting certain aspects of the transformer's attention mechanism at run-time. Demonstrated on two benchmark datasets, Breakfast and 50Salads, our approach outperforms current state-of-the-art methods for long-term action anticipation using short video context by up to 9%.
Code (0)
등록된 구현이 없습니다.
Tasks
Action AnticipationLong Term Action AnticipationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Guided Attention for Next Active Object @ EGO4D STA Challenge
In this technical report, we describe the Guided-Attention mechanism based solution for the short-term anticipation (STA) challenge for the EGO4D challenge. It combines the object detections, and the spatiotemporal featu…
ObjectShort-term Object Interaction AnticipationEnhancing Next Active Object-based Egocentric Action Anticipation with Guided Attention
Short-term action anticipation (STA) in first-person videos is a challenging task that involves understanding the next active object interactions and predicting future actions. Existing action anticipation methods have p…
Action AnticipationObjectShort-term Object Interaction AnticipationLeveraging Next-Active Objects for Context-Aware Anticipation in Egocentric Videos
Objects are crucial for understanding human-object interactions. By identifying the relevant objects, one can also predict potential future interactions or actions that may occur with these objects. In this paper, we stu…
Action AnticipationActive Object LocalizationHuman-Object Interaction DetectionObject+2Context-Enhanced Memory-Refined Transformer for Online Action Detection
Online Action Detection (OAD) detects actions in streaming videos using past observations. State-of-the-art OAD approaches model past observations and their interactions with an anticipated future. The past is encoded us…
Action DetectionDecoderOnline Action DetectionSemantically Guided Representation Learning For Action Anticipation
Action anticipation is the task of forecasting future activity from a partially observed sequence of events. However, this task is exposed to intrinsic future uncertainty and the difficulty of reasoning upon interconnect…
Action AnticipationRepresentation Learning