Graph-Based High-Order Relation Modeling for Long-Term Action Recognition
Long-term actions involve many important visual concepts, e.g., objects, motions, and sub-actions, and there are various relations among these concepts, which we call basic relations. These basic relations will jointly affect each other during the temporal evolution of long-term actions, which forms the high-order relations that are essential for long-term action recognition. In this paper, we propose a Graph-based High-order Relation Modeling (GHRM) module to exploit the high-order relations in the long-term actions for long-term action recognition. In GHRM, each basic relation in the long-term actions will be modeled by a graph, where each node represents a segment in a long video. Moreover, when modeling each basic relation, the information from all the other basic relations will be incorporated by GHRM, and thus the high-order relations in the long-term actions can be well exploited. To better exploit the high-order relations along the time dimension, we design a GHRM-layer consisting of a Temporal-GHRM branch and a Semantic-GHRM branch, which aims to model the local temporal high-order relations and global semantic high-order relations. The experimental results on three long-term action recognition datasets, namely, Breakfast, Charades, and MultiThumos, demonstrate the effectiveness of our model.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionLong-video Activity RecognitionRelationVideo ClassificationVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Hierarchical Attention Graph for Scientific Document Summarization in Global and Local Level
Scientific document summarization has been a challenging task due to the long structure of the input text. The long input hinders the simultaneous effective modeling of both global high-order relations between sentences …
Document SummarizationExtractive SummarizationScientific Document SummarizationSentenceCCMamba: Topologically-Informed Selective State-Space Networks on Combinatorial Complexes for Higher-Order Graph Learning
Topological deep learning has emerged as a powerful paradigm for modeling higher-order relational structures beyond pairwise interactions that standard graph neural networks fail to capture. While combinatorial complexes…
Graph LearningHigh-order Semantic Role Labeling
Semantic role labeling is primarily used to identify predicates, arguments, and their semantic relationships. Due to the limitations of modeling methods and the conditions of pre-identified predicates, previous work has …
Semantic Role LabelingVocal Bursts Intensity PredictionHEGEL: Hypergraph Transformer for Long Document Summarization
Extractive summarization for long documents is challenging due to the extended structured input context. The long-distance sentence dependency hinders cross-sentence relations modeling, the critical step of extractive su…
Document SummarizationExtractive SummarizationSentenceKNH: Multi-View Modeling with K-Nearest Hyperplanes Graph for Misinformation Detection
Graphs are one of the most efficacious structures for representing datapoints and their relations, and they have been largely exploited for different applications. Previously, the higher-order relations between the nodes…
ArticlesMisinformation