Cognitive resilience: Unraveling the proficiency of image-captioning models to interpret masked visual content
This study explores the ability of Image Captioning (IC) models to decode masked visual content sourced from diverse datasets. Our findings reveal the IC model's capability to generate captions from masked images, closely resembling the original content. Notably, even in the presence of masks, the model adeptly crafts descriptive textual information that goes beyond what is observable in the original image-generated captions. While the decoding performance of the IC model experiences a decline with an increase in the masked region's area, the model still performs well when important regions of the image are not masked at high coverage.
Code (1)
Tasks
DescriptiveImage CaptioningSimilar Papers 제목 키워드 기반
CNN Fixations: An unraveling approach to visualize the discriminative image regions
Deep convolutional neural networks (CNN) have revolutionized various fields of vision research and have seen unprecedented adoption for multiple tasks such as classification, detection, captioning, etc. However, they off…
Caption GenerationImage CaptioningObject RecognitionUnraveling Fundamental Properties of Power System Resilience Curves using Unsupervised Machine Learning
The standard model of infrastructure resilience, the resilience triangle, has been the primary way of characterizing and quantifying infrastructure resilience. However, the theoretical model merely provides a one-size-fi…
A Reconstruction-Based Framework for Caption Evaluation Beyond Reference Captions
Image captioning is a primary task in vision--language research, yet assessing how faithfully a caption preserves image semantics without relying on reference captions remains unsettled. Prevailing evaluations rely on hu…
Image CaptioningExploring Student Representation For Neural Cognitive Diagnosis
Cognitive diagnosis, the goal of which is to obtain the proficiency level of students on specific knowledge concepts, is an fundamental task in smart educational systems. Previous works usually represent each student as …
cognitive diagnosisBrain and Cognitive Reserve: Translation via Network Control Theory
Traditional approaches to understanding the brain's resilience to neuropathology have identified neurophysiological variables, often described as brain or cognitive 'reserve,' associated with better outcomes. However, me…
Translation