Paint4Poem: A Dataset for Artistic Visualization of Classical Chinese Poems
In this work we propose a new task: artistic visualization of classical Chinese poems, where the goal is to generatepaintings of a certain artistic style for classical Chinese poems. For this purpose, we construct a new dataset called Paint4Poem. Thefirst part of Paint4Poem consists of 301 high-quality poem-painting pairs collected manually from an influential modern Chinese artistFeng Zikai. As its small scale poses challenges for effectively training poem-to-painting generation models, we introduce the secondpart of Paint4Poem, which consists of 3,648 caption-painting pairs collected manually from Feng Zikai's paintings and 89,204 poem-painting pairs collected automatically from the web. We expect the former to help learning the artist painting style as it containshis most paintings, and the latter to help learning the semantic relevance between poems and paintings. Further, we analyze Paint4Poem regarding poem diversity, painting style, and the semantic relevance between poems and paintings. We create abenchmark for Paint4Poem: we train two representative text-to-image generation models: AttnGAN and MirrorGAN, and evaluate theirperformance regarding painting pictorial quality, painting stylistic relevance, and semantic relevance between poems and paintings.The results indicate that the models are able to generate paintings that have good pictorial quality and mimic Feng Zikai's style, but thereflection of poem semantics is limited. The dataset also poses many interesting research directions on this task, including transferlearning, few-shot learning, text-to-image generation for low-resource data etc. The dataset is publicly available.(https://github.com/paint4poem/paint4poem)
Code (1)
Tasks
Few-Shot LearningImage GenerationText to Image GenerationText-to-Image GenerationSimilar Papers 제목 키워드 기반
A Novel Approach to Artistic Textual Visualization via GAN
While the visualization of statistical data tends to a mature technology, the visualization of textual data is still in its infancy, especially for the artistic text. Due to the fact that visualization of artistic text i…
Generative Adversarial NetworkSemi-supervised Chinese Poem-to-Painting Generation via Cycle-consistent Adversarial Networks
Classical Chinese poetry and painting represent the epitome of artistic expression, but the abstract and symbolic nature of their relationship poses a significant challenge for computational translation. Most existing me…
DiversityHow Images Inspire Poems: Generating Classical Chinese Poetry from Images with Memory Networks
With the recent advances of neural models and natural language processing, automatic generation of classical Chinese poetry has drawn significant attention due to its artistic and cultural value. Previous works mainly fo…
DecoderRecognizing Art Style Automatically in painting with deep learning
The artistic style (or artistic movement) of a painting is a rich descriptor that captures both visual and historical information about the painting. Correctly identifying the artistic style of a paintings is crucial f…
Artistic style classificationDeep LearningOpt-In Art: Learning Art Styles Only from Few Examples
We explore whether pre-training on datasets with paintings is necessary for a model to learn an artistic style with only a few examples. To investigate this, we train a text-to-image model exclusively on photographs, wit…
Image GenerationText-to-Image Generation