Imagining from Images with an AI Storytelling Tool
A method for generating narratives by analyzing single images or image sequences is presented, inspired by the time immemorial tradition of Narrative Art. The proposed method explores the multimodal capabilities of GPT-4o to interpret visual content and create engaging stories, which are illustrated by a Stable Diffusion XL model. The method is supported by a fully implemented tool, called ImageTeller, which accepts images from diverse sources as input. Users can guide the narrative's development according to the conventions of fundamental genres - such as Comedy, Romance, Tragedy, Satire or Mystery -, opt to generate data-driven stories, or to leave the prototype free to decide how to handle the narrative structure. User interaction is provided along the generation process, allowing the user to request alternative chapters or illustrations, and even reject and restart the story generation based on the same input. Additionally, users can attach captions to the input images, influencing the system's interpretation of the visual content. Examples of generated stories are provided, along with details on how to access the prototype.
Code (0)
등록된 구현이 없습니다.
Tasks
Story GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
From Image Captioning to Visual Storytelling
Visual Storytelling is a challenging multimodal task between Vision & Language, where the purpose is to generate a story for a stream of images. Its difficulty lies on the fact that the story should be both grounded to t…
Visual StorytellingImage CaptioningFairyTailor: A Multimodal Generative Framework for Storytelling
Storytelling is an open-ended task that entails creative thinking and requires a constant flow of ideas. Natural language generation (NLG) for storytelling is especially challenging because it requires the generated text…
Story GenerationText GenerationWhere Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI Collaboration
Data storytelling is powerful for communicating data insights, but it requires diverse skills and considerable effort from human creators. Recent research has widely explored the potential for artificial intelligence (AI…
Where Do People Tell Stories Online? Story Detection Across Online Communities
Story detection in online communities is a challenging task as stories are scattered across communities and interwoven with non-storytelling spans within a single text. We address this challenge by building and releasing…
Persuasion StrategiesIncorporating Textual Evidence in Visual Storytelling
Previous work on visual storytelling mainly focused on exploring image sequence as evidence for storytelling and neglected textual evidence for guiding story generation. Motivated by human storytelling process which reca…
Object RecognitionStory GenerationVisual Storytelling