Rank Pooling for Action Recognition
We propose a function-based temporal pooling method that captures the latent structure of the video sequence data - e.g. how frame-level features evolve over time in a video. We show how the parameters of a function that has been fit to the video data can serve as a robust new video representation. As a specific example, we learn a pooling function via ranking machines. By learning to rank the frame-level features of a video in chronological order, we obtain a new representation that captures the video-wide temporal dynamics of a video, suitable for action recognition. Other than ranking functions, we explore different parametric models that could also explain the temporal changes in videos. The proposed functional pooling methods, and rank pooling in particular, is easy to interpret and implement, fast to compute and effective in recognizing a wide variety of actions. We evaluate our method on various benchmarks for generic action, fine-grained action and gesture recognition. Results show that rank pooling brings an absolute improvement of 7-10 average pooling baseline. At the same time, rank pooling is compatible with and complementary to several appearance and local motion based methods and features, such as improved trajectories and deep learning features.
Code (1)
Tasks
Action RecognitionGesture RecognitionLearning-To-RankTemporal Action LocalizationSimilar Papers 제목 키워드 기반
Discriminative Hierarchical Rank Pooling for Activity Recognition
We present hierarchical rank pooling, a video sequence encoding method for activity recognition. It consists of a network of rank pooling functions which captures the dynamics of rich convolutional neural network feature…
Action RecognitionActivity RecognitionTemporal Action LocalizationDynamic Image Networks for Action Recognition
We introduce the concept of dynamic image, a novel compact representation of videos useful for video analysis especially when convolutional neural networks (CNNs) are used. The dynamic image is based on the rank pooling …
Action RecognitionTemporal Action LocalizationDiscriminatively Learned Hierarchical Rank Pooling Networks
In this work, we present novel temporal encoding methods for action and activity classification by extending the unsupervised rank pooling temporal encoding method in two ways. First, we present "discriminative rank pool…
Activity RecognitionBilevel OptimizationGeneral ClassificationEigen Evolution Pooling for Human Action Recognition
We introduce Eigen Evolution Pooling, an efficient method to aggregate a sequence of feature vectors. Eigen evolution pooling is designed to produce compact feature representations for a sequence of feature vectors, whil…
Action RecognitionTemporal Action LocalizationGeneralized Rank Pooling for Activity Recognition
Most popular deep models for action recognition split video sequences into short sub-sequences consisting of a few frames; frame-based features are then pooled for recognizing the activity. Usually, this pooling step dis…
Action RecognitionActivity RecognitionRiemannian optimizationTemporal Action Localization