홈 › Papers

Visual Robustness Benchmark for Visual Question Answering (VQA)

2024-07-03 · Md Farhan Ishmam, Ishmam Tashdeed, Talukder Asir Saadat, Md Hamjajul Ashmafee, Abu Raihan Mostofa Kamal, Md. Azam Hossain

Can Visual Question Answering (VQA) systems perform just as well when deployed in the real world? Or are they susceptible to realistic corruption effects e.g. image blur, which can be detrimental in sensitive applications, such as medical VQA? While linguistic or textual robustness has been thoroughly explored in the VQA literature, there has yet to be any significant work on the visual robustness of VQA models. We propose the first large-scale benchmark comprising 213,000 augmented images, challenging the visual robustness of multiple VQA models and assessing the strength of realistic visual corruptions. Additionally, we have designed several robustness evaluation metrics that can be aggregated into a unified metric and tailored to fit a variety of use cases. Our experiments reveal several insights into the relationships between model size, performance, and robustness with the visual corruptions. Our benchmark highlights the need for a balanced approach in model development that considers model performance without compromising the robustness.

📄 PDF Abstract BibTeX arXiv:2407.03386

Code (1)

ishmamt/Visual-Robustness 공식 구현 pytorch

Tasks

Visual Question Answering Visual Question Answering (VQA)

Methods 이 논문이 사용한 방법론

BLIP Vision-Language Pre-training (VLP) has advanced the performance for many vision-language tasks. However, most existing pre-trained models only excel in either understanding-based…

ViLT ViLT is a minimal vision-and-language pre-training transformer model where processing of visual inputs is simplified to just the same convolution-free manner that text inputs are…

Vision And Text Transformer For Predicting Answerability On Visual Question Answering

2026-09-15 · Tung Le, Huy Tien Nguyen, Le Minh Nguyen arxiv

Answerability on Visual Question Answering is a novel and attractive task to predict answerable scores between images and questions in multi-modal data. Existing works often utilize a binary mapping from visual question …

Visual Question Answering

Speech-Based Visual Question Answering

2017-05-01 · Ted Zhang, Dengxin Dai, Tinne Tuytelaars, Marie-Francine Moens 외

This paper introduces speech-based visual question answering (VQA), the task of generating an answer given an image and a spoken question. Two methods are studied: an end-to-end, deep neural network that directly uses au…

Automatic Speech Recognition Automatic Speech Recognition (ASR)Question Answering speech-recognition+3

Are Large Vision Language Models Truly Grounded in Medical Images? Evidence from Italian Clinical Visual Question Answering

2025-11-24 · Federico Felizzi, Olivia Riccomi, Michele Ferramola, Francesco Andrea Causio 외 arxiv

Large vision language models (VLMs) have achieved impressive performance on medical visual question answering benchmarks, yet their reliance on visual information remains unclear. We investigate whether frontier VLMs dem…

Visual Question Answering Visual Grounding

Learning from Lexical Perturbations for Consistent Visual Question Answering

2020-11-26 · Spencer Whitehead, Hui Wu, Yi Ren Fung, Heng Ji 외

Existing Visual Question Answering (VQA) models are often fragile and sensitive to input variations. In this paper, we propose a novel approach to address this issue based on modular networks, which creates two questions…

Question Answering Visual Question Answering Visual Question Answering (VQA)Visual Reasoning

VQA-LOL: Visual Question Answering under the Lens of Logic

2020-02-19 · ECCV 2020 8 · Tejas Gokhale, Pratyay Banerjee, Chitta Baral, Yezhou Yang

Logical connectives and their implications on the meaning of a natural language sentence are a fundamental aspect of understanding. In this paper, we investigate whether visual question answering (VQA) systems trained to…

Negation Question Answering Sentence Visual Question Answering+1