Query-controllable Video Summarization
When video collections become huge, how to explore both within and across videos efficiently is challenging. Video summarization is one of the ways to tackle this issue. Traditional summarization approaches limit the effectiveness of video exploration because they only generate one fixed video summary for a given input video independent of the information need of the user. In this work, we introduce a method which takes a text-based query as input and generates a video summary corresponding to it. We do so by modeling video summarization as a supervised learning problem and propose an end-to-end deep learning based method for query-controllable video summarization to generate a query-dependent video summary. Our proposed method consists of a video summary controller, video summary generator, and video summary output module. To foster the research of query-controllable video summarization and conduct our experiments, we introduce a dataset that contains frame-based relevance score labels. Based on our experimental result, it shows that the text-based query helps control the video summary. It also shows the text-based query improves our model performance. Our code and dataset: https://github.com/Jhhuangkay/Query-controllable-Video-Summarization.
Code (1)
Tasks
Video SummarizationSimilar Papers 제목 키워드 기반
Prompts to Summaries: Zero-Shot Language-Guided Video Summarization
The explosive growth of video data intensified the need for flexible user-controllable summarization tools that can operate without domain-specific training data. Existing methods either rely on datasets, limiting genera…
GPUQuery focused video summarizationVideo SummarizationQuery-Focused Extractive Video Summarization
Video data is explosively growing. As a result of the "big video data", intelligent algorithms for automatic video summarization have re-emerged as a pressing need. We develop a probabilistic model, Sequential and Hierar…
Query focused video summarizationVideo SummarizationCausal Video Summarizer for Video Exploration
Recently, video summarization has been proposed as a method to help video exploration. However, traditional video summarization models only generate a fixed video summary which is usually independent of user-specific nee…
DecoderVideo SummarizationCLIP-It! Language-Guided Video Summarization
A generic video summary is an abridged version of a video that conveys the whole story and features the most important scenes. Yet the importance of scenes in a video is often subjective, and users should have the option…
Query-focused SummarizationQuery focused video summarizationSupervised Video SummarizationVideo SummarizationQuery-Aware Sparse Coding for Multi-Video Summarization
Given the explosive growth of online videos, it is becoming increasingly important to relieve the tedious work of browsing and managing the video content of interest. Video summarization aims at providing such a techniqu…
Video Summarization