Initialization Strategies of Spatio-Temporal Convolutional Neural Networks
We propose a new way of incorporating temporal information present in videos into Spatial Convolutional Neural Networks (ConvNets) trained on images, that avoids training Spatio-Temporal ConvNets from scratch. We describe several initializations of weights in 3D Convolutional Layers of Spatio-Temporal ConvNet using 2D Convolutional Weights learned from ImageNet. We show that it is important to initialize 3D Convolutional Weights judiciously in order to learn temporal representations of videos. We evaluate our methods on the UCF-101 dataset and demonstrate improvement over Spatial ConvNets.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Convolutional State Space Models for Long-Range Spatiotemporal Modeling
Effectively modeling long spatiotemporal sequences is challenging due to the need to model complex spatial correlations and long-range temporal dependencies simultaneously. ConvLSTMs attempt to address this by updating t…
MinecraftState Space ModelsSpatiotemporal Fusion in 3D CNNs: A Probabilistic View
Despite the success in still image recognition, deep neural networks for spatiotemporal signal tasks (such as human action recognition in videos) still suffers from low efficacy and inefficiency over the past years. Rece…
Action RecognitionAction Recognition In VideosTemporal Action LocalizationSpatiotemporal Pyramid Network for Video Action Recognition
Two-stream convolutional networks have shown strong performance in video action recognition tasks. The key idea is to learn spatiotemporal features by fusing convolutional networks spatially and temporally. However, it r…
Action RecognitionTemporal Action LocalizationSpatio-Temporal Vegetation Pixel Classification By Using Convolutional Networks
Plant phenology studies rely on long-term monitoring of life cycles of plants. High-resolution unmanned aerial vehicles (UAVs) and near-surface technologies have been used for plant monitoring, demanding the creation of …
ClassificationGeneral ClassificationLocal Spatiotemporal Convolutional Network for Robust Gait Recognition
Gait recognition, as a promising biometric technology, identifies individuals through their unique walking patterns and offers distinctive advantages including non-invasiveness, long-range applicability, and resistance t…
Gait Recognition