Transformers in Action Recognition: A Review on Temporal Modeling
In vision-based action recognition, spatio-temporal features from different modalities are used for recognizing activities. Temporal modeling is a long challenge of action recognition. However, there are limited methods such as pre-computed motion features, three-dimensional (3D) filters, and recurrent neural networks (RNN) for modeling motion information in deep-based approaches. Recently, transformers success in modeling long-range dependencies in natural language processing (NLP) tasks has gotten great attention from other domains; including speech, image, and video, to rely entirely on self-attention without using sequence-aligned RNNs or convolutions. Although the application of transformers to action recognition is relatively new, the amount of research proposed on this topic within the last few years is astounding. This paper especially reviews recent progress in deep learning methods for modeling temporal variations. It focuses on action recognition methods that use transformers for temporal modeling, discussing their main features, used modalities, and identifying opportunities and challenges for future research.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionSimilar Papers 제목 키워드 기반
Vision Transformers for Action Recognition: A Survey
Vision transformers are emerging as a powerful tool to solve computer vision problems. Recent techniques have also proven the efficacy of transformers beyond the image domain to solve numerous video-related tasks. Among …
Action RecognitionDimensionality ReductionSurveyTemporal Action Localization+1Automatic Speech Recognition with BERT and CTC Transformers: A Review
This review paper provides a comprehensive analysis of recent advances in automatic speech recognition (ASR) with bidirectional encoder representations from transformers BERT and connectionist temporal classification (CT…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionSpatiotemporal Self-attention Modeling with Temporal Patch Shift for Action Recognition
Transformer-based methods have recently achieved great advancement on 2D image-based vision tasks. For 3D video-based tasks such as action recognition, however, directly applying spatiotemporal transformers on video data…
Action ClassificationAction RecognitionLate Temporal Modeling in 3D CNN Architectures with BERT for Action Recognition
In this work, we combine 3D convolution with late temporal modeling for action recognition. For this aim, we replace the conventional Temporal Global Average Pooling (TGAP) layer at the end of 3D convolutional architectu…
Action RecognitionSimba: Mamba augmented U-ShiftGCN for Skeletal Action Recognition in Videos
Skeleton Action Recognition (SAR) involves identifying human actions using skeletal joint coordinates and their interconnections. While plain Transformers have been attempted for this task, they still fall short compared…
Action RecognitionAction Recognition In VideosMamba