Dual-CNN: A Convolutional language decoder for paragraph image captioning
Abstract The task of paragraph image captioning aims to generate a coherent paragraph describing a given image. However, due to their limited ability to capture long-term dependency, recurrent neural network or long-short term memory based decoders could hardly generate satisfactory textual descriptions with a long paragraph. In addition, the training inefficiency in the sequential decoders is significantly observed. Motivated by the advantage of convolutional neural network (i.e., CNN), in this paper, we propose a Dual-CNN decoder with long-term memory ability and parallel computation, which can produce a semantically coherent paragraph for an image. Our Dual-CNN model is evaluated on the Stanford image-paragraph dataset. Extensive experiments demonstrate that our Dual-CNN achieves comparable results compared with state-of-the-art models. Furthermore, the diversity and coherence of generated paragraphs are analyzed to show the superiority of our approach.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderDiversityImage CaptioningImage Paragraph CaptioningSimilar Papers 제목 키워드 기반
Improving Diversity and Reducing Redundancy in Paragraph Captions
The purpose of an image paragraph captioning model is to produce detailed descriptions of the source images. Generally, paragraph captioning models use encoder-decoder based architectures similar to the standard image…
DecoderDense CaptioningDiversityImage Captioning+1Byte-Level Recursive Convolutional Auto-Encoder for Text
This article proposes to auto-encode text at byte-level using convolutional networks with a recursive architecture. The motivation is to explore whether it is possible to have scalable and homogeneous text generation at …
DecoderText GenerationText Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise
In this paper, we introduce a novel dIffusion language modEl pre-training framework for text generation, which we call GENIE. GENIE is a large-scale pretrained diffusion language model that consists of an encoder and a d…
DecoderDenoisingLanguage ModelingLanguage Modelling+1Hierarchical Scene Graph Encoder-Decoder for Image Paragraph Captioning
When we humans tell a long paragraph about an image, we usually first implicitly compose a mental “script” and then comply with it to generate the paragraph. Inspired by this, we render the modern encoder-decoder base…
DecoderImage Paragraph CaptioningSentenceConvolutional Auto-encoding of Sentence Topics for Image Paragraph Generation
Image paragraph generation is the task of producing a coherent story (usually a paragraph) that describes the visual content of an image. The problem nevertheless is not trivial especially when there are multiple descrip…
DescriptiveImage Paragraph CaptioningSentencevalid