Learnable pooling with Context Gating for video classification
Current methods for video analysis often extract frame-level features using pre-trained convolutional neural networks (CNNs). Such features are then aggregated over time e.g., by simple temporal averaging or more sophisticated recurrent neural networks such as long short-term memory (LSTM) or gated recurrent units (GRU). In this work we revise existing video representations and study alternative methods for temporal aggregation. We first explore clustering-based aggregation layers and propose a two-stream architecture aggregating audio and visual features. We then introduce a learnable non-linear unit, named Context Gating, aiming to model interdependencies among network activations. Our experimental results show the advantage of both improvements for the task of video classification. In particular, we evaluate our method on the large-scale multi-modal Youtube-8M v2 dataset and outperform all other methods in the Youtube 8M Large-Scale Video Understanding challenge.
Code (5)
Tasks
ClassificationClusteringGeneral ClassificationVideo ClassificationVideo UnderstandingSimilar Papers 제목 키워드 기반
Learnable Pooling Methods for Video Classification
We introduce modifications to state-of-the-art approaches to aggregating local video descriptors by using attention mechanisms and function approximations. Rather than using ensembles of existing architectures, we provid…
ClassificationGeneral ClassificationVideo ClassificationVideo UnderstandingEnhancing Hyperspace Analogue to Language (HAL) Representations via Attention-Based Pooling for Text Classification
The Hyperspace Analogue to Language (HAL) model relies on global word co-occurrence matrices to construct distributional semantic representations. While these representations capture lexical relationships effectively, ag…
Text ClassificationSentiment AnalysisLearning to Recognize Actions on Objects in Egocentric Video with Attention Dictionaries
We present EgoACO, a deep neural architecture for video action recognition that learns to pool action-context-object descriptors from frame level features by leveraging the verb-noun structure of action labels in egocent…
Action RecognitionObjectTemporal Action LocalizationA Fully Trainable Network with RNN-based Pooling
Pooling is an important component in convolutional neural networks (CNNs) for aggregating features and reducing computational burden. Compared with other components such as convolutional layers and fully connected layers…
Advanced Feature Learning on Point Clouds using Multi-resolution Features and Learnable Pooling
Existing point cloud feature learning networks often incorporate sequences of sampling, neighborhood grouping, neighborhood-wise feature learning, and feature aggregation to learn high-semantic point features that repres…
3D Point Cloud Classification