GLAC Net: GLocal Attention Cascading Networks for Multi-image Cued Story Generation
The task of multi-image cued story generation, such as visual storytelling dataset (VIST) challenge, is to compose multiple coherent sentences from a given sequence of images. The main difficulty is how to generate image-specific sentences within the context of overall images. Here we propose a deep learning network model, GLAC Net, that generates visual stories by combining global-local (glocal) attention and context cascading mechanisms. The model incorporates two levels of attention, i.e., overall encoding level and image feature level, to construct image-dependent sentences. While standard attention configuration needs a large number of parameters, the GLAC Net implements them in a very simple way via hard connections from the outputs of encoders or image features onto the sentence generators. The coherency of the generated story is further improved by conveying (cascading) the information of the previous sentence to the next sentence serially. We evaluate the performance of the GLAC Net on the visual storytelling dataset (VIST) and achieve very competitive results compared to the state-of-the-art techniques. Our code and pre-trained models are available here.
Code (1)
Tasks
SentenceStory GenerationVisual StorytellingSimilar Papers 제목 키워드 기반
3DGTN: 3D Dual-Attention GLocal Transformer Network for Point Cloud Classification and Segmentation
Although the application of Transformers in 3D point cloud processing has achieved significant progress and success, it is still challenging for existing 3D Transformer methods to efficiently and accurately learn both va…
ClassificationDecoderPoint Cloud ClassificationSegmentationAMD-HookNet for Glacier Front Segmentation
Knowledge on changes in glacier calving front positions is important for assessing the status of glaciers. Remote sensing imagery provides the ideal database for monitoring calving front positions, however, it is not fea…
Calving Front Delineation In Synthetic Aperture Radar ImageryCalving Front Delineation In Synthetic Aperture Radar Imagery With Fixed Training AmountSegmentationGlocalFuse-Depth: Fusing Transformers and CNNs for All-day Self-supervised Monocular Depth Estimation
In recent years, self-supervised monocular depth estimation has drawn much attention since it frees of depth annotations and achieved remarkable results on standard benchmarks. However, most of existing methods only focu…
AllDepth EstimationMonocular Depth EstimationGlocalCLIP: Object-agnostic Global-Local Prompt Learning for Zero-shot Anomaly Detection
Zero-shot anomaly detection (ZSAD) is crucial for detecting anomalous patterns in target datasets without using training samples, specifically in scenarios where there are distributional differences between the target do…
Anomaly DetectionContrastive LearningPrompt Learningzero-shot anomaly detectionGlacier Calving Front Segmentation Using Attention U-Net
An essential climate variable to determine the tidewater glacier status is the location of the calving front position and the separation of seasonal variability from long-term trends. Previous studies have proposed deep …