Comprehensive Video Understanding: Video summarization with content-based video recommender design
Video summarization aims to extract keyframes/shots from a long video. Previous methods mainly take diversity and representativeness of generated summaries as prior knowledge in algorithm design. In this paper, we formulate video summarization as a content-based recommender problem, which should distill the most useful content from a long video for users who suffer from information overload. A scalable deep neural network is proposed on predicting if one video segment is a useful segment for users by explicitly modelling both segment and video. Moreover, we accomplish scene and action recognition in untrimmed videos in order to find more correlations among different aspects of video understanding tasks. Also, our paper will discuss the effect of audio and visual features in summarization task. We also extend our work by data augmentation and multi-task learning for preventing the model from early-stage overfitting. The final results of our model win the first place in ICCV 2019 CoView Workshop Challenge Track.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionData AugmentationDiversityMulti-Task LearningVideo SummarizationVideo UnderstandingSimilar Papers 제목 키워드 기반
Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization
Emotion is a key element in user-generated videos. However, it is difficult to understand emotions conveyed in such videos due to the complex and unstructured nature of user-generated content and the sparsity of video fr…
Emotion RecognitionTransfer LearningVideo Emotion RecognitionZero-Shot LearningLanguage-guided Recursive Spatiotemporal Graph Modeling for Video Summarization
Video summarization aims to select keyframes that are visually diverse and can represent the whole story of a given video. Previous approaches have focused on global interlinkability between frames in a video by temporal…
Video SummarizationPersonalized Video Summarization by Multimodal Video Understanding
Video summarization techniques have been proven to improve the overall user experience when it comes to accessing and comprehending video content. If the user's preference is known, video summarization can identify signi…
Unsupervised Video SummarizationVideo SummarizationVideo UnderstandingVideo Summarization Techniques: A Comprehensive Review
The rapid expansion of video content across a variety of industries, including social media, education, entertainment, and surveillance, has made video summarization an essential field of study. The current work is a sur…
Abstractive Text SummarizationExtractive SummarizationVideo SummarizationRealizing Video Summarization from the Path of Language-based Semantic Understanding
The recent development of Video-based Large Language Models (VideoLLMs), has significantly advanced video summarization by aligning video features and, in some cases, audio features with Large Language Models (LLMs). Eac…
Mixture-of-ExpertsVideo GenerationVideo Summarization