Generating coherent comic with rich story using ChatGPT and Stable Diffusion
Past work demonstrated that using neural networks, we can extend unfinished music pieces while maintaining the music style of the musician. With recent advancements in large language models and diffusion models, we are now capable of generating comics with an interesting storyline while maintaining the art style of the artist. In this paper, we used ChatGPT to generate storylines and dialogue and then generated the comic using stable diffusion. We introduced a novel way to evaluate AI-generated stories, and we achieved SOTA performance on character fidelity and art style by fine-tuning stable diffusion using LoRA, ControlNet, etc.
Code (0)
등록된 구현이 없습니다.
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
From Panels to Prose: Generating Literary Narratives from Comics
Comics have long been a popular form of storytelling, offering visually engaging narratives that captivate audiences worldwide. However, the visual nature of comics presents a significant barrier for visually impaired re…
Optical Character Recognition (OCR)Toward accessible comics for blind and low vision readers
This work explores how to fine-tune large language models using prompt engineering techniques with contextual information for generating an accurate text description of the full story, ready to be forwarded to off-the-sh…
Optical Character RecognitionPrompt EngineeringSpeech SynthesisComics for Everyone: Generating Accessible Text Descriptions for Comic Strips
Comic strips are a popular and expressive form of visual storytelling that can convey humor, emotion, and information. However, they are inaccessible to the BLV (Blind or Low Vision) community, who cannot perceive the im…
Language ModelingLanguage ModellingLarge Language ModelMultimodal Large Language Model+1Puzzled By ChatGPT? No more! A Jigsaw Puzzle to Promote AI Literacy and Awareness
The rapid adoption of Generative AI, including LLM-based chatbots like ChatGPT, has highlighted the need for accessible ways to support public understanding and AI literacy. To address this need, we introduce a game-base…
Visual StorytellingManga Generation via Layout-controllable Diffusion
Generating comics through text is widely studied. However, there are few studies on generating multi-panel Manga (Japanese comics) solely based on plain text. Japanese manga contains multiple panels on a single page, wit…
Semantic correspondence