Representation Flow for Action Recognition
In this paper, we propose a convolutional layer inspired by optical flow algorithms to learn motion representations. Our representation flow layer is a fully-differentiable layer designed to capture the flow' of any representation channel within a convolutional neural network for action recognition. Its parameters for iterative flow optimization are learned in an end-to-end fashion together with the other CNN model parameters, maximizing the action recognition performance. Furthermore, we newly introduce the concept of learning flow of flow' representations by stacking multiple representation flow layers. We conducted extensive experimental evaluations, confirming its advantages over previous recognition models using traditional optical flows in both computational speed and performance. Code/models available here: https://piergiaj.github.io/rep-flow-site/
Code (5)
Tasks
Action ClassificationAction RecognitionAction Recognition In VideosActivity RecognitionActivity Recognition In VideosOptical Flow EstimationTemporal Action LocalizationVideo ClassificationVideo UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Scene Flow to Action Map: A New Representation for RGB-D based Action Recognition with Convolutional Neural Networks
Scene flow describes the motion of 3D objects in real world and potentially could be the basis of a good feature for 3D action recognition. However, its use for action recognition, especially in the context of convolutio…
3D Action RecognitionAction RecognitionTemporal Action LocalizationAn End-to-End Two-Stream Network Based on RGB Flow and Representation Flow for Human Action Recognition
With the rapid advancements in deep learning, computer vision tasks have seen significant improvements, making two-stream neural networks a popular focus for video based action recognition. Traditional models using RGB a…
Action RecognitionOptical Flow EstimationTemporal Action LocalizationCombining Spatio-Temporal Appearance Descriptors and Optical Flow for Human Action Recognition in Video Data
This paper proposes combining spatio-temporal appearance (STA) descriptors with optical flow for human action recognition. The STA descriptors are local histogram-based descriptors of space-time, suitable for building a …
Action RecognitionOptical Flow EstimationTemporal Action LocalizationActionFlowNet: Learning Motion Representation for Action Recognition
Even with the recent advances in convolutional neural networks (CNN) in various visual recognition tasks, the state-of-the-art action recognition system still relies on hand crafted motion feature such as optical flow to…
Action RecognitionOptical Flow EstimationTemporal Action LocalizationOptical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition
Motion representation plays a vital role in human action recognition in videos. In this study, we introduce a novel compact motion representation for video action recognition, named Optical Flow guided Feature (OFF), whi…
Action RecognitionAction Recognition In VideosOptical Flow EstimationTemporal Action Localization