MASTAF: A Model-Agnostic Spatio-Temporal Attention Fusion Network for Few-shot Video Classification

2021-12-08 · Rex Liu, Huanle Zhang, Hamed Pirsiavash, Xin Liu

We propose MASTAF, a Model-Agnostic Spatio-Temporal Attention Fusion network for few-shot video classification. MASTAF takes input from a general video spatial and temporal representation,e.g., using 2D CNN, 3D CNN, and Video Transformer. Then, to make the most of such representations, we use self- and cross-attention models to highlight the critical spatio-temporal region to increase the inter-class variations and decrease the intra-class variations. Last, MASTAF applies a lightweight fusion network and a nearest neighbor classifier to classify each query video. We demonstrate that MASTAF improves the state-of-the-art performance on three few-shot video classification benchmarks(UCF101, HMDB51, and Something-Something-V2), e.g., by up to 91.6%, 69.5%, and 60.7% for five-way one-shot video classification, respectively.

📄 PDF Abstract BibTeX arXiv:2112.04585

Code (1)

lxhit/STAF 공식 구현 pytorch

Tasks

Classification Video Classification

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

3D CNN 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Adam 설명 없음

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Position-Wise Feed-Forward Layer 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Version

2022-03-29 · Razvan-Gabriel Cirstea, Bin Yang, Chenjuan Guo, Tung Kieu 외

Traffic time series forecasting is challenging due to complex spatio-temporal dynamics time series from different locations often have distinct patterns; and for the same time series, patterns may vary across time, where…

Time Series Time Series Analysis Time Series Forecasting

Spatiotemporal Imputation with Graph-Informed Flow Matching

2026-06-04 · Zepeng Zhang, Aref Einizade, Jhony H. Giraldo, Olga Fink arxiv

Missing data is a common challenge in spatiotemporal systems, arising in applications such as air quality monitoring and urban traffic management. Traditional machine learning approaches, like recurrent and graph neural …

Co-Fusion4D: Spatio-temporal Collaborative Fusion for Robust 3D Object Detection

2026-05-19 · Wenxuan Li, Qin Zou, Shoubing Chen, Chi Chen 외 arxiv

In autonomous driving, 3D object detection is essential for accurate perception and reliable decision-making. However, object motion and ego-motion often induce cross-frame spatiotemporal inconsistencies in BEV-based det…

Robust 3D Object Detection Autonomous Driving

Generative AI Enables EEG Super-Resolution via Spatio-Temporal Adaptive Diffusion Learning

2024-07-03 · Shuqiang Wang, Tong Zhou, Yanyan Shen, Ye Li 외

Electroencephalogram (EEG) technology, particularly high-density EEG (HD EEG) devices, is widely used in fields such as neuroscience. HD EEG devices improve the spatial resolution of EEG by placing more electrodes on the…

Denoising Diagnostic EEG Electroencephalogram (EEG)+1

Towards Unifying Diffusion Models for Probabilistic Spatio-Temporal Graph Learning

2023-10-26 · Junfeng Hu, Xu Liu, Zhencheng Fan, Yuxuan Liang 외

Spatio-temporal graph learning is a fundamental problem in modern urban systems. Existing approaches tackle different tasks independently, tailoring their models to unique task characteristics. These methods, however, fa…

Denoising Graph Learning