PIC: Permutation Invariant Convolution for Recognizing Long-range Activities
Neural operations as convolutions, self-attention, and vector aggregation are the go-to choices for recognizing short-range actions. However, they have three limitations in modeling long-range activities. This paper presents PIC, Permutation Invariant Convolution, a novel neural layer to model the temporal structure of long-range activities. It has three desirable properties. i. Unlike standard convolution, PIC is invariant to the temporal permutations of features within its receptive field, qualifying it to model the weak temporal structures. ii. Different from vector aggregation, PIC respects local connectivity, enabling it to learn long-range temporal abstractions using cascaded layers. iii. In contrast to self-attention, PIC uses shared weights, making it more capable of detecting the most discriminant visual evidence across long and noisy videos. We study the three properties of PIC and demonstrate its effectiveness in recognizing the long-range activities of Charades, Breakfast, and MultiThumos.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Recognizing Multi-talker Speech with Permutation Invariant Training
In this paper, we propose a novel technique for direct recognition of multiple speech streams given the single channel of mixed speech, without first separating them. Our technique is based on permutation invariant train…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)speech-recognitionSpeech RecognitionFew-shot Action Recognition with Permutation-invariant Attention
Many few-shot learning models focus on recognising images. In contrast, we tackle a challenging task of few-shot action recognition from videos. We build on a C3D encoder for spatio-temporal video blocks to capture short…
Action RecognitionFew-Shot action recognitionFew Shot Action RecognitionFew-Shot Learning+2SPI-GCN: A Simple Permutation-Invariant Graph Convolutional Network
A wide range of machine learning problems involve handling graph-structured data. Existing machine learning approaches for graphs, however, often imply computing expensive graph similarity measures, preprocessing input g…
BIG-bench Machine LearningDeep LearningGeneral ClassificationGraph Classification+1Classification of Periodic Variable Stars with Novel Cyclic-Permutation Invariant Neural Networks
Neural networks (NNs) have been shown to be competitive against state-of-the-art feature engineering and random forest (RF) classification of periodic variable stars. Although previous work utilising NNs has made use of …
AstronomyFeature EngineeringGeneral ClassificationTime Series AnalysisPermutation-equivariant neural networks applied to dynamics prediction
The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the u…
PredictionTranslation