Learning Video-Story Composition via Recurrent Neural Network
In this paper, we propose a learning-based method to compose a video-story from a group of video clips that describe an activity or experience. We learn the coherence between video clips from real videos via the Recurrent Neural Network (RNN) that jointly incorporates the spatial-temporal semantics and motion dynamics to generate smooth and relevant compositions. We further rearrange the results generated by the RNN to make the overall video-story compatible with the storyline structure via a submodular ranking optimization process. Experimental results on the video-story dataset show that the proposed algorithm outperforms the state-of-the-art approach.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Long History Short-Term Memory for Long-Term Video Prediction
While video prediction approaches have advanced considerably in recent years, learning to predict long-term future is challenging — ambiguous future or error propagation over time yield blurry predictions. To address thi…
Video PredictionStorytelling of Photo Stream with Bidirectional Multi-thread Recurrent Neural Network
Visual storytelling aims to generate human-level narrative language (i.e., a natural paragraph with multiple sentences) from a photo streams. A typical photo story consists of a global timeline with multi-thread local st…
Video CaptioningVisual StorytellingVideo Storytelling: Textual Summaries for Events
Bridging vision and natural language is a longstanding goal in computer vision and multimedia research. While earlier works focus on generating a single-sentence description for visual content, recent works have studied …
DiversityReinforcement LearningSentenceVideo-Story Composition via Plot Analysis
We address the problem of composing a story out of multiple short video clips taken by a person during an activity or experience. Inspired by plot analysis of written stories, our method generates a sequence of video cli…
Optical Flow EstimationPatch MatchingHistory-Guided Video Diffusion
Classifier-free guidance (CFG) is a key technique for improving conditional generation in diffusion models, enabling more accurate control while enhancing sample quality. It is natural to extend this technique to video d…
Video Generation