Incorporating Textual Evidence in Visual Storytelling
Previous work on visual storytelling mainly focused on exploring image sequence as evidence for storytelling and neglected textual evidence for guiding story generation. Motivated by human storytelling process which recalls stories for familiar images, we exploit textual evidence from similar images to help generate coherent and meaningful stories. To pick the images which may provide textual experience, we propose a two-step ranking method based on image object recognition techniques. To utilize textual information, we design an extended Seq2Seq model with two-channel encoder and attention. Experiments on the VIST dataset show that our method outperforms state-of-the-art baseline models without heavy engineering.
Code (0)
등록된 구현이 없습니다.
Tasks
Object RecognitionStory GenerationVisual StorytellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Context-aware Visual Storytelling with Visual Prefix Tuning and Contrastive Learning
Visual storytelling systems generate multi-sentence stories from image sequences. In this task, capturing contextual information and bridging visual variation bring additional challenges. We propose a simple yet effectiv…
Contrastive LearningInformativenessSentenceVisual StorytellingContextualStory: Consistent Visual Storytelling with Spatially-Enhanced and Storyline Context
Visual storytelling involves generating a sequence of coherent frames from a textual storyline while maintaining consistency in characters and scenes. Existing autoregressive methods, which rely on previous frame-sentenc…
Image GenerationStory ContinuationStory VisualizationText-to-Image Generation+1AESOP: Abstract Encoding of Stories, Objects, and Pictures
Visual storytelling and story comprehension are uniquely human skills that play a central role in how we learn about and experience the world. Despite remarkable progress in recent years in synthesis of visual and te…
Story CompletionVisual StorytellingVIST-GPT: Ushering in the Era of Visual Storytelling with LLMs?
Visual storytelling is an interdisciplinary field combining computer vision and natural language processing to generate cohesive narratives from sequences of images. This paper presents a novel approach that leverages re…
Visual GroundingVisual StorytellingA Pipeline for Creative Visual Storytelling
Computational visual storytelling produces a textual description of events and interpretations depicted in a sequence of images. These texts are made possible by advances and cross-disciplinary approaches in natural lang…
Visual Storytelling