BERT for Large-scale Video Segment Classification with Test-time Augmentation
This paper presents our approach to the third YouTube-8M video understanding competition that challenges par-ticipants to localize video-level labels at scale to the pre-cise time in the video where the label actually occurs. Ourmodel is an ensemble of frame-level models such as GatedNetVLAD and NeXtVLAD and various BERT models withtest-time augmentation. We explore multiple ways to ag-gregate BERT outputs as video representation and variousways to combine visual and audio information. We proposetest-time augmentation as shifting video frames to one leftor right unit, which adds variety to the predictions and em-pirically shows improvement in evaluation metrics. We firstpre-train the model on the 4M training video-level data, andthen fine-tune the model on 237K annotated video segment-level data. We achieve MAP@100K 0.7871 on private test-ing video segment data, which is ranked 9th over 283 teams.
Code (0)
등록된 구현이 없습니다.
Tasks
General ClassificationVideo UnderstandingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ALBERT: Advanced Localization and Bidirectional Encoder Representations from Transformers for Automotive Damage Evaluation
This paper introduces ALBERT, an instance segmentation model specifically designed for comprehensive car damage and part segmentation. Leveraging the power of Bidirectional Encoder Representations, ALBERT incorporates ad…
Instance SegmentationSegmentationSemantic Segmentation1st Place Solution for 5th LSVOS Challenge: Referring Video Object Segmentation
The recent transformer-based models have dominated the Referring Video Object Segmentation (RVOS) task due to the superior performance. Most prior works adopt unified DETR framework to generate segmentation masks in quer…
ObjectReferring Video Object SegmentationSegmentationSemantic Segmentation+2VideoChain: A Transformer-Based Framework for Multi-hop Video Question Generation
Multi-hop Question Generation (QG) effectively evaluates reasoning but remains confined to text; Video Question Generation (VideoQG) is limited to zero-hop questions over single segments. To address this, we introduce Vi…
Semantic SimilarityQuestion GenerationTemporal Deformable Residual Networks for Action Segmentation in Videos
This paper is about temporal segmentation of human actions in videos. We introduce a new model -- temporal deformable residual network (TDRN) -- aimed at analyzing video intervals at multiple temporal scales for labelin…
Action SegmentationSegmentationMEmoBERT: Pre-training Model with Prompt-based Learning for Multimodal Emotion Recognition
Multimodal emotion recognition study is hindered by the lack of labelled corpora in terms of scale and diversity, due to the high annotation cost and label ambiguity. In this paper, we propose a pre-training model \textb…
DiversityEmotion ClassificationEmotion RecognitionMultimodal Emotion Recognition+1