WEmbSim: A Simple yet Effective Metric for Image Captioning
The area of automatic image caption evaluation is still undergoing intensive research to address the needs of generating captions which can meet adequacy and fluency requirements. Based on our past attempts at developing highly sophisticated learning-based metrics, we have discovered that a simple cosine similarity measure using the Mean of Word Embeddings(MOWE) of captions can actually achieve a surprisingly high performance on unsupervised caption evaluation. This inspires our proposed work on an effective metric WEmbSim, which beats complex measures such as SPICE, CIDEr and WMD at system-level correlation with human judgments. Moreover, it also achieves the best accuracy at matching human consensus scores for caption pairs, against commonly used unsupervised methods. Therefore, we believe that WEmbSim sets a new baseline for any complex metric to be justified.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningWord EmbeddingsSimilar Papers 제목 키워드 기반
Towards Unique and Informative Captioning of Images
Despite considerable progress, state of the art image captioning models produce generic captions, leaving out important image details. Furthermore, these systems may even misrepresent the image in order to produce a simp…
DiversityImage CaptioningRe-RankingA Frustratingly Simple Approach for End-to-End Image Captioning
Image Captioning is a fundamental task to join vision and language, concerning about cross-modal understanding and text generation. Recent years witness the emerging attention on image captioning. Most of existing works …
DecoderImage CaptioningObjectText GenerationMeasuring directional bias amplification in image captions using predictability
When we train models on biased ML datasets, they not only learn these biases but can inflate them at test time - a phenomenon called bias amplification. To measure bias amplification in ML datasets, many co-occurrence-ba…
Image Captioningimage-classificationImage ClassificationDistinctive Image Captioning via CLIP Guided Group Optimization
Image captioning models are usually trained according to human annotated ground-truth captions, which could generate accurate but generic captions. In this paper, we focus on generating distinctive captions that can dist…
Image CaptioningActor-Critic Sequence Training for Image Captioning
Generating natural language descriptions of images is an important capability for a robot or other visual-intelligence driven AI agent that may need to communicate with human users about what it is seeing. Such image cap…
AI AgentImage Captioningreinforcement-learningReinforcement Learning+1