Video Summarization: Towards Entity-Aware Captions
Existing popular video captioning benchmarks and models deal with generic captions devoid of specific person, place or organization named entities. In contrast, news videos present a challenging setting where the caption requires such named entities for meaningful summarization. As such, we propose the task of summarizing news video directly to entity-aware captions. We also release a large-scale dataset, VIEWS (VIdeo NEWS), to support research on this task. Further, we propose a method that augments visual information from videos with context retrieved from external world knowledge to generate entity-aware captions. We demonstrate the effectiveness of our approach on three video captioning models. We also show that our approach generalizes to existing news image captions dataset. With all the extensive experiments and insights, we believe we establish a solid basis for future research on this challenging task.
Code (1)
Tasks
Image CaptioningVideo CaptioningVideo SummarizationWorld KnowledgeSimilar Papers 제목 키워드 기반
Identity-Aware Human-Object Interaction Motion Captioning
Existing human-object interaction (HOI) motion captioning methods typically describe what happens while referring to the subject using generic terms such as "a person" or "someone", without grounding the caption in subje…
Motion CaptioningLanguage-Guided Self-Supervised Video Summarization Using Text Semantic Matching Considering the Diversity of the Video
Current video summarization methods rely heavily on supervised computer vision techniques, which demands time-consuming and subjective manual annotations. To overcome these limitations, we investigated self-supervised vi…
DiversitySupervised Video SummarizationVideo SummarizationVideo Summarization with Large Language Models
The exponential increase in video content poses significant challenges in terms of efficient navigation, search, and retrieval, thus requiring advanced video summarization techniques. Existing video summarization methods…
Large Language ModelVideo SummarizationVideo Paragraph Captioning as a Text Summarization Task
Video paragraph captioning aims to generate a set of coherent sentences to describe a video that contains several events. Most previous methods simplify this task by using ground-truth event segments. In this work, we pr…
SentenceText SummarizationERA: Entity Relationship Aware Video Summarization with Wasserstein GAN
Video summarization aims to simplify large scale video browsing by generating concise, short summaries that diver from but well represent the original video. Due to the scarcity of video annotations, recent progress for …
Unsupervised Video SummarizationVideo Summarization