NewsNet: A Novel Dataset for Hierarchical Temporal Segmentation
Temporal video segmentation is the get-to-go automatic video analysis, which decomposes a long-form video into smaller components for the following-up understanding tasks. Recent works have studied several levels of granularity to segment a video, such as shot, event, and scene. Those segmentations can help compare the semantics in the corresponding scales, but lack a wider view of larger temporal spans, especially when the video is complex and structured. Therefore, we present two abstractive levels of temporal segmentations and study their hierarchy to the existing fine-grained levels. Accordingly, we collect NewsNet, the largest news video dataset consisting of 1,000 videos in over 900 hours, associated with several tasks for hierarchical temporal video segmentation. Each news video is a collection of stories on different topics, represented as aligned audio, visual, and textual data, along with extensive frame-wise annotations in four granularities. We assert that the study on NewsNet can advance the understanding of complex structured video and benefit more areas such as short-video creation, personalized advertisement, digital instruction, and education. Our dataset and code is publicly available at: https://github.com/NewsNet-Benchmark/NewsNet.
Code (0)
등록된 구현이 없습니다.
Tasks
Video SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
NewsNet-SDF: Stochastic Discount Factor Estimation with Pretrained Language Model News Embeddings via Adversarial Networks
Stochastic Discount Factor (SDF) models provide a unified framework for asset pricing and risk assessment, yet traditional formulations struggle to incorporate unstructured textual information. We introduce NewsNet-SDF, …
ArticlesLanguage ModelingLanguage ModellingMultimodal Deep Learning+1Unsupervised Skeleton-Based Action Segmentation via Hierarchical Spatiotemporal Vector Quantization
We propose a novel hierarchical spatiotemporal vector quantization framework for unsupervised skeleton-based temporal action segmentation. We first introduce a hierarchical approach, which includes two consecutive levels…
Action SegmentationRefining Action Segmentation With Hierarchical Video Representations
In this paper, we propose Hierarchical Action Segmentation Refiner (HASR), which can refine temporal action segmentation results from various models by understanding the overall context of a given video in a hierarch…
Action SegmentationSegmentationTemporal Action SegmentationA Heuristic-driven Uncertainty based Ensemble Framework for Fake News Detection in Tweets and News Articles
The significance of social media has increased manifold in the past few decades as it helps people from even the most remote corners of the world to stay connected. With the advent of technology, digital media has become…
ArticlesFake News DetectionTemporal-aware Hierarchical Mask Classification for Video Semantic Segmentation
Modern approaches have proved the huge potential of addressing semantic segmentation as a mask classification task which is widely used in instance-level segmentation. This paradigm trains models by assigning part of obj…
ClassificationDecoderSegmentationSemantic Segmentation+1