LOGO-Former: Local-Global Spatio-Temporal Transformer for Dynamic Facial Expression Recognition
Previous methods for dynamic facial expression recognition (DFER) in the wild are mainly based on Convolutional Neural Networks (CNNs), whose local operations ignore the long-range dependencies in videos. Transformer-based methods for DFER can achieve better performances but result in higher FLOPs and computational costs. To solve these problems, the local-global spatio-temporal Transformer (LOGO-Former) is proposed to capture discriminative features within each frame and model contextual relationships among frames while balancing the complexity. Based on the priors that facial muscles move locally and facial expressions gradually change, we first restrict both the space attention and the time attention to a local window to capture local interactions among feature tokens. Furthermore, we perform the global attention by querying a token with features from each local window iteratively to obtain long-range information of the whole video sequence. In addition, we propose the compact loss regularization term to further encourage the learned features have the minimum intra-class distance and the maximum inter-class distance. Experiments on two in-the-wild dynamic facial expression datasets (i.e., DFEW and FERV39K) indicate that our method provides an effective way to make use of the spatial and temporal dependencies for DFER.
Code (0)
등록된 구현이 없습니다.
Tasks
Dynamic Facial Expression RecognitionFacial Expression RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Logo-LLM: Local and Global Modeling with Large Language Models for Time Series Forecasting
Time series forecasting is critical across multiple domains, where time series data exhibits both local patterns and global dependencies. While Transformer-based methods effectively capture global dependencies, they ofte…
Time SeriesTime Series ForecastingHierarchical Graph Convolutional Skeleton Transformer for Action Recognition
Graph convolutional networks (GCNs) have emerged as dominant methods for skeleton-based action recognition. However, they still suffer from two problems, namely, neighborhood constraints and entangled spatiotemporal feat…
Action RecognitionDisentanglementSkeleton Based Action RecognitionBiC-Net: Learning Efficient Spatio-Temporal Relation for Text-Video Retrieval
The task of text-video retrieval aims to understand the correspondence between language and vision, has gained increasing attention in recent years. Previous studies either adopt off-the-shelf 2D/3D-CNN and then use aver…
Cross-Modal RetrievalRelationRetrievalVideo Retrieval+1PoseMamba: Monocular 3D Human Pose Estimation with Bidirectional Global-Local Spatio-Temporal State Space Model
Transformers have significantly advanced the field of 3D human pose estimation (HPE). However, existing transformer-based methods primarily use self-attention mechanisms for spatio-temporal modeling, leading to a quadrat…
3D Human Pose EstimationLong-range modelingMambaMonocular 3D Human Pose Estimation+1UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learning
It is a challenging task to learn rich and multi-scale spatiotemporal semantics from high-dimensional videos, due to large local redundancy and complex global dependency between video frames. The recent advances in this …
Representation Learning