Tencent AVS: A Holistic Ads Video Dataset for Multi-modal Scene Segmentation
Temporal video segmentation and classification have been advanced greatly by public benchmarks in recent years. However, such research still mainly focuses on human actions, failing to describe videos in a holistic view. In addition, previous research tends to pay much attention to visual information yet ignores the multi-modal nature of videos. To fill this gap, we construct the Tencent Ads Video Segmentation'~(TAVS) dataset in the ads domain to escalate multi-modal video analysis to a new level. TAVS describes videos from three independent perspectives as presentation form', place', and style', and contains rich multi-modal information such as video, audio, and text. TAVS is organized hierarchically in semantic aspects for comprehensive temporal video segmentation with three levels of categories for multi-label classification, e.g., place' - working place' - `office'. Therefore, TAVS is distinguished from previous temporal segmentation datasets due to its multi-modal information, holistic view of categories, and hierarchical granularities. It includes 12,000 videos, 82 classes, 33,900 segments, 121,100 shots, and 168,500 labels. Accompanied with TAVS, we also present a strong multi-modal video segmentation baseline coupled with multi-label class prediction. Extensive experiments are conducted to evaluate our proposed method as well as existing representative methods to reveal key challenges of our dataset TAVS.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONScene SegmentationSegmentationVideo SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Tencent-MVSE: A Large-Scale Benchmark Dataset for Multi-Modal Video Similarity Evaluation
Multi-modal video similarity evaluation is important for video recommendation systems such as video de-duplication, relevance matching, ranking, and diversity control. However, there still lacks a benchmark dataset t…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DiversityRecommendation Systems+3Overview of Tencent Multi-modal Ads Video Understanding Challenge
Multi-modal Ads Video Understanding Challenge is the first grand challenge aiming to comprehensively understand ads videos. Our challenge includes two tasks: video structuring in the temporal dimension and multi-modal vi…
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONVideo ClassificationVideo UnderstandingVideo Ads Content Structuring by Combining Scene Confidence Prediction and Tagging
Video ads segmentation and tagging is a challenging task due to two main reasons: (1) the video scene structure is complex and (2) it includes multiple modalities (e.g., visual, audio, text.). While previous work focuses…
TAGHunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding
We present Hunyuan-DiT, a text-to-image diffusion transformer with fine-grained understanding of both English and Chinese. To construct Hunyuan-DiT, we carefully design the transformer structure, text encoder, and positi…
Image GenerationLanguage ModelingLanguage ModellingLarge Language Model+2Tencent Video Dataset (TVD): A Video Dataset for Learning-based Visual Data Compression and Analysis
Learning-based visual data compression and analysis have attracted great interest from both academia and industry recently. More training as well as testing datasets, especially good quality video datasets are highly des…
4kData Compressionobject-detectionObject Detection