A Circular Window-based Cascade Transformer for Online Action Detection
Online action detection aims at the accurate action prediction of the current frame based on long historical observations. Meanwhile, it demands real-time inference on online streaming videos. In this paper, we advocate a novel and efficient principle for online action detection. It merely updates the latest and oldest historical representations in one window but reuses the intermediate ones, which have been already computed. Based on this principle, we introduce a window-based cascade Transformer with a circular historical queue, where it conducts multi-stage attentions and cascade refinement on each window. We also explore the association between online action detection and its counterpart offline action segmentation as an auxiliary task. We find that such an extra supervision helps discriminative history clustering and acts as feature augmentation for better training the classifier and cascade refinement. Our proposed method achieves the state-of-the-art performances on three challenging datasets THUMOS'14, TVSeries, and HDD. Codes will be available after acceptance.
Code (0)
등록된 구현이 없습니다.
Tasks
Action DetectionAction SegmentationOnline Action DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
von Mises Tapering: A Circular Data Windowing
Continuous standard windowing is revisited and a new taper shape is introduced, which is based on the normal circular distribution by von Mises. Continuous-time windows are considered and their spectra obtained. A brief …
BolT: Fused Window Transformers for fMRI Time Series Analysis
Deep-learning models have enabled performance leaps in analysis of high-dimensional functional MRI (fMRI) data. Yet, many previous methods are suboptimally sensitive for contextual representations across diverse time sca…
Time SeriesTime Series AnalysisVon Mises Tapering: A New Circular Windowing
Discrete and continuous standard windowing are revisited and a a new taper is introduced, which is derived from the normal circular distribution by von Mises. Both the continuous-time and the discrete-time windows are co…
Cascaded Boundary Regression for Temporal Action Detection
Temporal action detection in long videos is an important problem. State-of-the-art methods address this problem by applying action classifiers on sliding windows. Although sliding windows may contain an identifiable port…
Action DetectionregressionLong Short-Term Transformer for Online Action Detection
We present Long Short-term TRansformer (LSTR), a temporal modeling algorithm for online action detection, which employs a long- and short-term memory mechanism to model prolonged sequence data. It consists of an LSTR enc…
Action DetectionDecoderOnline Action DetectionPlaying the Game of 2048