Action Selection Learning for Multi-label Multi-view Action Recognition
Multi-label multi-view action recognition aims to recognize multiple concurrent or sequential actions from untrimmed videos captured by multiple cameras. Existing work has focused on multi-view action recognition in a narrow area with strong labels available, where the onset and offset of each action are labeled at the frame-level. This study focuses on real-world scenarios where cameras are distributed to capture a wide-range area with only weak labels available at the video-level. We propose the method named Multi-view Action Selection Learning (MultiASL), which leverages action selection learning to enhance view fusion by selecting the most useful information from different viewpoints. The proposed method includes a Multi-view Spatial-Temporal Transformer video encoder to extract spatial and temporal features from multi-viewpoint videos. Action Selection Learning is employed at the frame-level, using pseudo ground-truth obtained from weak labels at the video-level, to identify the most relevant frames for action recognition. Experiments in a real-world office environment using the MM-Office dataset demonstrate the superior performance of the proposed method compared to existing methods. The source code is available at https://github.com/thanhhff/MultiASL/.
Code (1)
Tasks
Action RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Active View Selection for Scene-level Multi-view Crowd Counting and Localization with Limited Labeling Budget
Multi-view crowd counting and localization fuse the input multi-views for estimating the crowd number or locations on the ground. Existing methods mainly focus on accurately predicting on the crowd shown in the input vie…
Crowd CountingAdaptive Collaborative Sot Label Learning for Unsupervised Multi-view Feature Selection
Unsupervised multi-view feature selection aims to select informative features with multi-view features and unsupervised learning. It is a challenging problem due to the absence of explicit semantic supervision. Recently…
feature selectiongraph constructionPseudo LabelRedundancy-optimized Multi-head Attention Networks for Multi-View Multi-Label Feature Selection
Multi-view multi-label data offers richer perspectives for artificial intelligence, but simultaneously presents significant challenges for feature selection due to the inherent complexity of interrelations among features…
Joint Learning of Unsupervised Multi-view Feature and Instance Co-selection with Cross-view Imputation
Feature and instance co-selection, which aims to reduce both feature dimensionality and sample size by identifying the most informative features and instances, has attracted considerable attention in recent years. Howeve…
Multi-view Unsupervised Feature Selection by Cross-diffused Matrix Alignment
Multi-view high-dimensional data become increasingly popular in the big data era. Feature selection is a useful technique for alleviating the curse of dimensionality in multi-view learning. In this paper, we study unsupe…
feature selectionMULTI-VIEW LEARNING