Continuous Action Recognition Based on Sequence Alignment
Continuous action recognition is more challenging than isolated recognition because classification and segmentation must be simultaneously carried out. We build on the well known dynamic time warping (DTW) framework and devise a novel visual alignment technique, namely dynamic frame warping (DFW), which performs isolated recognition based on per-frame representation of videos, and on aligning a test sequence with a model sequence. Moreover, we propose two extensions which enable to perform recognition concomitant with segmentation, namely one-pass DFW and two-pass DFW. These two methods have their roots in the domain of continuous recognition of speech and, to the best of our knowledge, their extension to continuous visual action recognition has been overlooked. We test and illustrate the proposed techniques with a recently released dataset (RAVEL) and with two public-domain datasets widely used in action recognition (Hollywood-1 and Hollywood-2). We also compare the performances of the proposed isolated and continuous recognition algorithms with several recently published methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionDynamic Time WarpingSegmentationTemporal Action LocalizationSimilar Papers 제목 키워드 기반
A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training
This work develops a continuous sign language (SL) recognition framework with deep neural networks, which directly transcribes videos of SL sentences to sequences of ordered gloss labels. Previous methods dealing with…
Optical Flow EstimationSign Language RecognitionRecurrent Convolutional Neural Networks for Continuous Sign Language Recognition by Staged Optimization
This work presents a weakly supervised framework with deep neural networks for vision-based continuous sign language recognition, where the ordered gloss labels but no exact temporal locations are available with the vide…
SentenceSign Language RecognitionIterative Alignment Network for Continuous Sign Language Recognition
In this paper, we propose an alignment network with iterative optimization for weakly supervised continuous sign language recognition. Our framework consists of two modules: a 3D convolutional residual network (3D-ResNet…
DecoderDynamic Time WarpingGeneral ClassificationSign Language RecognitionRe-Sign: Re-Aligned End-To-End Sequence Modelling With Deep Recurrent CNN-HMMs
This work presents an iterative re-alignment approach applicable to visual sequence labelling tasks such as gesture recognition, activity recognition and continuous sign language recognition. Previous methods dealing wi…
Activity RecognitionGesture RecognitionSign Language RecognitionSMART: MLLM-guided Temporal Alignment for Unifying Sign Language Recognition and Spotting
Continuous sign language recognition (CSLR) aims to recognize gloss sequences from unsegmented sign videos under weak sequence-level supervision. However, existing methods rely on sentence-level gloss annotations, provid…
Sign Language RecognitionRepresentation Learning