Efficient Action Localization with Approximately Normalized Fisher Vectors
The Fisher vector (FV) representation is a high-dimensional extension of the popular bag-of-word representation. Transformation of the FV by power and L2 normalizations has shown to significantly improve its performance, and led to state-of-the-art results for a range of image and video classification and retrieval tasks. These normalizations, however, render the representation non-additive over local descriptors. Combined with its high dimensionality, this makes the FV computationally expensive for the purpose of localization tasks. In this paper we present approximations to both these normalizations, which yield significant improvements in the memory and computational costs of the FV when used for localization. Second, we show how these approximations can be used to define upper-bounds on the score function that can be efficiently evaluated, which enables the use of branch-and-bound search as an alternative to exhaustive sliding window search. We present experimental evaluation results on classification and temporal localization of actions in videos. These show that the our approximations lead to a speedup of at least one order of magnitude, while maintaining state-of-the-art action recognition and localization performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Action LocalizationAction RecognitionGeneral ClassificationRetrievalTemporal Action LocalizationTemporal LocalizationVideo ClassificationSimilar Papers 제목 키워드 기반
Adversarial Fisher Vectors for Unsupervised Representation Learning
We examine Generative Adversarial Networks (GANs) through the lens of deep Energy Based Models (EBMs), with the goal of exploiting the density model that follows from this formulation. In contrast to a traditional view w…
General ClassificationRepresentation LearningHyper-Fisher Vectors for Action Recognition
In this paper, a novel encoding scheme combining Fisher vector and bag-of-words encodings has been proposed for recognizing action in videos. The proposed Hyper-Fisher vector encoding is sum of local Fisher vectors which…
Action RecognitionTemporal Action LocalizationDINO as a von Mises-Fisher mixture model
Self-distillation methods using Siamese networks are popular for self-supervised pre-training. DINO is one such method based on a cross-entropy loss between $K$-dimensional probability vectors, obtained by applying a sof…
modelApproximate Spectral Decomposition of Fisher Information Matrix for Simple ReLU Networks
We argue the Fisher information matrix (FIM) of one hidden layer networks with the ReLU activation function. For a network, let $W$ denote the $d \times p$ weight matrix from the $d$-dimensional input to the hidden layer…
The geometry of kernelized spectral clustering
Clustering of data sets is a standard problem in many areas of science and engineering. The method of spectral clustering is based on embedding the data set using a kernel function, and using the top eigenvectors of the …
Clustering