Knowledge Distillation for Human Action Anticipation
We consider the task of training a neural network to anticipate human actions in video. This task is challenging given the complexity of video data, the stochastic nature of the future, and the limited amount of annotated training data. In this paper, we propose a novel knowledge distillation framework that uses an action recognition network to supervise the training of an action anticipation network, guiding the latter to attend to the relevant information needed for correctly anticipating the future actions. This framework is possible thanks to a novel loss function to account for positional shifts of semantic concepts in a dynamic video. The knowledge distillation framework is a form of self-supervised learning, and it takes advantage of unlabeled data. Experimental results on JHMDB and EPIC-KITCHENS dataset show the effectiveness of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
Action AnticipationAction RecognitionKnowledge DistillationSelf-Supervised LearningTemporal Action LocalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Cross-modal Contrastive Distillation for Instructional Activity Anticipation
In this study, we aim to predict the plausible future action steps given an observation of the past and study the task of instructional activity anticipation. Unlike previous anticipation tasks that aim at action label p…
Knowledge DistillationText-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation
Anticipating future actions in a video is useful for many autonomous and assistive technologies. Most prior action anticipation work treat this as a vision modality problem, where the models learn the task information pr…
Action AnticipationTransfer LearningKnowledge Distillation for Action Anticipation via Label Smoothing
Human capability to anticipate near future from visual observations and non-verbal cues is essential for developing intelligent systems that need to interact with people. Several research areas, such as human-robot inter…
Action AnticipationAutonomous DrivingKnowledge DistillationMissing Labels1st Place Solution to the EPIC-Kitchens Action Anticipation Challenge 2022
In this report, we describe the technical details of our submission to the EPIC-Kitchens Action Anticipation Challenge 2022. In this competition, we develop the following two approaches. 1) Anticipation Time Knowledge Di…
Action AnticipationKnowledge DistillationStreaming egocentric action anticipation: An evaluation scheme and approach
Egocentric action anticipation aims to predict the future actions the camera wearer will perform from the observation of the past. While predictions about the future should be available before the predicted events take p…
Action AnticipationKnowledge Distillation