MASS: Overcoming Language Bias in Image-Text Matching
Pretrained visual-language models have made significant advancements in multimodal tasks, including image-text retrieval. However, a major challenge in image-text matching lies in language bias, where models predominantly rely on language priors and neglect to adequately consider the visual content. We thus present Multimodal ASsociation Score (MASS), a framework that reduces the reliance on language priors for better visual accuracy in image-text matching problems. It can be seamlessly incorporated into existing visual-language models without necessitating additional training. Our experiments have shown that MASS effectively lessens language bias without losing an understanding of linguistic compositionality. Overall, MASS offers a promising solution for enhancing image-text matching performance in visual-language models.
Code (0)
등록된 구현이 없습니다.
Tasks
Image-text matchingImage-text RetrievalMultimodal AssociationRetrievalText MatchingText RetrievalSimilar Papers 제목 키워드 기반
Refining Skewed Perceptions in Vision-Language Models through Visual Representations
Large vision-language models (VLMs), such as CLIP, have become foundational, demonstrating remarkable success across a variety of downstream tasks. Despite their advantages, these models, akin to other foundational syste…
MisconceptionsOvercoming Language Bias in Remote Sensing Visual Question Answering via Adversarial Training
The Visual Question Answering (VQA) system offers a user-friendly interface and enables human-computer interaction. However, VQA models commonly face the challenge of language bias, resulting from the learned superficial…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search
Internet search affects people's cognition of the world, so mitigating biases in search results and learning fair models is imperative for social good. We study a unique gender bias in image search in this work: the sear…
Image RetrievalNatural Language QueriesX-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models
This paper introduces a novel explainable image quality evaluation approach called X-IQE, which leverages visual large language models (LLMs) to evaluate text-to-image generation methods by generating textual explanation…
BenchmarkingImage GenerationText to Image GenerationText-to-Image GenerationWomen also Snowboard: Overcoming Bias in Captioning Models (Extended Abstract)
Most machine learning methods are known to capture and exploit biases of the training data. While some biases are beneficial for learning, others are harmful. Specifically, image captioning models tend to exaggerate bias…
Image Captioning