V4D:4D Convolutional Neural Networks for Video-level Representation Learning
Most existing 3D CNNs for video representation learning are clip-based methods, and thus do not consider video-level temporal evolution of spatio-temporal features. In this paper, we propose Video-level 4D Convolutional Neural Networks, referred as V4D, to model the evolution of long-range spatio-temporal representation with 4D convolutions, and at the same time, to preserve strong 3D spatio-temporal representation with residual connections. Specifically, we design a new 4D residual block able to capture inter-clip interactions, which could enhance the representation power of the original clip-level 3D CNNs. The 4D residual blocks can be easily integrated into the existing 3D CNNs to perform long-range modeling hierarchically. We further introduce the training and inference methods for the proposed V4D. Extensive experiments are conducted on three video recognition benchmarks, where V4D achieves excellent results, surpassing recent 3D CNNs by a large margin.
Code (1)
Tasks
Long-range modelingRepresentation LearningVideo RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
V4D: 4D Convolutional Neural Networks for Video-level Representation Learning
Most existing 3D CNN structures for video representation learning are clip-based methods, and do not consider video-level temporal evolution of spatio-temporal features. In this paper, we propose Video-level 4D Convoluti…
Representation LearningVideo RecognitionF4D: Factorized 4D Convolutional Neural Network for Efficient Video-level Representation Learning
Recent studies have shown that video-level representation learning is crucial to the capture and understanding of the long-range temporal structure for video action recognition. Most existing 3D convolutional neural netw…
Action RecognitionRepresentation LearningTemporal Action LocalizationOrder-aware Convolutional Pooling for Video Based Action Recognition
Most video based action recognition approaches create the video-level representation by temporally pooling the features extracted at each frame. The pooling methods that they adopt, however, usually completely or partial…
Action RecognitionTemporal Action LocalizationDelving Deeper into Convolutional Networks for Learning Video Representations
We propose an approach to learn spatio-temporal features in videos from intermediate visual representations we call "percepts" using Gated-Recurrent-Unit Recurrent Networks (GRUs).Our method relies on percepts that are e…
Action RecognitionDecoderTemporal Action LocalizationVideo CaptioningMHSCNet: A Multimodal Hierarchical Shot-aware Convolutional Network for Video Summarization
Video summarization intends to produce a concise video summary by effectively capturing and combining the most informative parts of the whole content. Existing approaches for video summarization regard the task as a fram…
Video Summarization