Learning-based Composite Metrics for Improved Caption Evaluation
The evaluation of image caption quality is a challenging task, which requires the assessment of two main aspects in a caption: adequacy and fluency. These quality aspects can be judged using a combination of several linguistic features. However, most of the current image captioning metrics focus only on specific linguistic facets, such as the lexical or semantic, and fail to meet a satisfactory level of correlation with human judgements at the sentence-level. We propose a learning-based framework to incorporate the scores of a set of lexical and semantic metrics as features, to capture the adequacy and fluency of captions at different linguistic levels. Our experimental results demonstrate that composite metrics draw upon the strengths of stand-alone measures to yield improved correlation and accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningLanguage ModelingLanguage ModellingSemantic Textual SimilaritySentenceSimilar Papers 제목 키워드 기반
LCEval: Learned Composite Metric for Caption Evaluation
Automatic evaluation metrics hold a fundamental importance in the development and fine-grained analysis of captioning systems. While current evaluation metrics tend to achieve an acceptable correlation with human judgeme…
SentenceFLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model
Most existing image captioning evaluation metrics focus on assigning a single numerical score to a caption by comparing it with reference captions. However, these methods do not provide an explanation for the assigned sc…
Image CaptioningLLM-Free Image Captioning Evaluation in Reference-Flexible Settings
We focus on the automatic evaluation of image captions in both reference-based and reference-free settings. Existing metrics based on large language models (LLMs) favor their own generations; therefore, the neutrality is…
Image CaptioningDENEB: A Hallucination-Robust Automatic Evaluation Metric for Image Captioning
In this work, we address the challenge of developing automatic evaluation metrics for image captioning, with a particular focus on robustness against hallucinations. Existing metrics are often inadequate for handling hal…
HallucinationImage CaptioningSemantic Compositions Enhance Vision-Language Contrastive Learning
In the field of vision-language contrastive learning, models such as CLIP capitalize on matched image-caption pairs as positive examples and leverage within-batch non-matching pairs as negatives. This approach has led to…
ClassificationContrastive LearningCross-Modal Retrievalimage-classification+6