Temporal Modeling Approaches for Large-scale Youtube-8M Video Understanding
This paper describes our solution for the video recognition task of the Google Cloud and YouTube-8M Video Understanding Challenge that ranked the 3rd place. Because the challenge provides pre-extracted visual and audio features instead of the raw videos, we mainly investigate various temporal modeling approaches to aggregate the frame-level features for multi-label video recognition. Our system contains three major components: two-stream sequence model, fast-forward sequence model and temporal residual neural networks. Experiment results on the challenging Youtube-8M dataset demonstrate that our proposed temporal modeling approaches can significantly improve existing temporal modeling approaches in the large-scale video recognition tasks. To be noted, our fast-forward LSTM with a depth of 7 layers achieves 82.75% in term of GAP@20 on the Kaggle Public test set.
Code (1)
Tasks
Video RecognitionVideo UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning to Localize Temporal Events in Large-scale Video Data
We address temporal localization of events in large-scale video data, in the context of the Youtube-8M Segments dataset. This emerging field within video recognition can enable applications to identify the precise time a…
Temporal LocalizationVideo RecognitionLarge-Scale YouTube-8M Video Understanding with Deep Neural Networks
Video classification problem has been studied many years. The success of Convolutional Neural Networks (CNN) in image recognition tasks gives a powerful incentive for researchers to create more advanced video classificat…
ClassificationGeneral ClassificationMixture-of-ExpertsVideo Classification+1YouTube-VOS: Sequence-to-Sequence Video Object Segmentation
Learning long-term spatial-temporal features are critical for many video analysis tasks. However, existing video segmentation methods predominantly rely on static image segmentation techniques, and methods capturing temp…
Image SegmentationObjectOne-shot visual object segmentationOptical Flow Estimation+7YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark
Learning long-term spatial-temporal features are critical for many video analysis tasks. However, existing video segmentation methods predominantly rely on static image segmentation techniques, and methods capturing temp…
Image SegmentationObjectOptical Flow EstimationSegmentation+4Video Instance Segmentation via Multi-scale Spatio-temporal Split Attention Transformer
State-of-the-art transformer-based video instance segmentation (VIS) approaches typically utilize either single-scale spatio-temporal features or per-frame multi-scale features during the attention computations. We argue…
Instance SegmentationSemantic SegmentationSTSVideo Instance Segmentation