Explicit Bias Discovery in Visual Question Answering Models

2018-11-19 · CVPR 2019 6 · Varun Manjunatha, Nirat Saini, Larry S. Davis

Researchers have observed that Visual Question Answering (VQA) models tend to answer questions by learning statistical biases in the data. For example, their answer to the question "What is the color of the grass?" is usually "Green", whereas a question like "What is the title of the book?" cannot be answered by inferring statistical biases. It is of interest to the community to explicitly discover such biases, both for understanding the behavior of such models, and towards debugging them. Our work address this problem. In a database, we store the words of the question, answer and visual words corresponding to regions of interest in attention maps. By running simple rule mining algorithms on this database, we discover human-interpretable rules which give us unique insight into the behavior of such models. Our results also show examples of unusual behaviors learned by models in attempting VQA tasks.

📄 PDF Abstract BibTeX arXiv:1811.07789

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering Visual Question Answering Visual Question Answering (VQA)

Transfer Learning via Unsupervised Task Discovery for Visual Question Answering

2018-10-03 · CVPR 2019 6 · Hyeonwoo Noh, Tae-hoon Kim, Jonghwan Mun, Bohyung Han

We study how to leverage off-the-shelf visual and linguistic data to cope with out-of-vocabulary answers in visual question answering task. Existing large-scale visual datasets with annotations such as image class labels…

Question Answering Transfer Learning Visual Question Answering Visual Question Answering (VQA)

Structure Causal Models and LLMs Integration in Medical Visual Question Answering

2025-05-05 · Zibo Xu, Qiang Li, Weizhi Nie, Weijie Wang 외

Medical Visual Question Answering (MedVQA) aims to answer medical questions according to medical images. However, the complexity of medical data leads to confounders that are difficult to observe, so bias between images …

Causal Inference Medical Visual Question Answering Question Answering Visual Question Answering

Toward Multi-Granularity Decision-Making: Explicit Visual Reasoning with Hierarchical Knowledge

2023-01-01 · ICCV 2023 1 · Yifeng Zhang, Shi Chen, Qi Zhao

Answering visual questions requires the ability to parse visual observations and correlate them with a variety of knowledge. Existing visual question answering (VQA) models either pay little attention to the role of …

Decision Making Question Answering Visual Question Answering Visual Question Answering (VQA)+1

Debiased Visual Question Answering from Feature and Sample Perspectives

2021-12-01 · NeurIPS 2021 12 · Zhiquan Wen, Guanghui Xu, Mingkui Tan, Qingyao Wu 외

Visual question answering (VQA) is designed to examine the visual-textual reasoning ability of an intelligent agent. However, recent observations show that many VQA models may only capture the biases between questions an…

Bias Detection Question Answering Visual Question Answering Visual Question Answering (VQA)

Interpretable Visual Question Answering via Reasoning Supervision

2023-09-07 · Maria Parelli, Dimitrios Mallis, Markos Diomataris, Vassilis Pitsikalis

Transformer-based architectures have recently demonstrated remarkable performance in the Visual Question Answering (VQA) task. However, such models are likely to disregard crucial visual cues and often rely on multimodal…

Common Sense Reasoning Question Answering Visual Grounding Visual Question Answering+1