Eliminating the Language Bias for Visual Question Answering with fine-grained Causal Intervention
Despite the remarkable advancements in Visual Question Answering (VQA), the challenge of mitigating the language bias introduced by textual information remains unresolved. Previous approaches capture language bias from a coarse-grained perspective. However, the finer-grained information within a sentence, such as context and keywords, can result in different biases. Due to the ignorance of fine-grained information, most existing methods fail to sufficiently capture language bias. In this paper, we propose a novel causal intervention training scheme named CIBi to eliminate language bias from a finer-grained perspective. Specifically, we divide the language bias into context bias and keyword bias. We employ causal intervention and contrastive learning to eliminate context bias and improve the multi-modal representation. Additionally, we design a new question-only branch based on counterfactual generation to distill and eliminate keyword bias. Experimental results illustrate that CIBi is applicable to various VQA models, yielding competitive performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningcounterfactualQuestion AnsweringSentenceVisual Question AnsweringVisual Question Answering (VQA)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Eliminating Catastrophic Interference with Biased Competition
We present here a model to take advantage of the multi-task nature of complex datasets by learning to separate tasks and subtasks in and end to end manner by biasing competitive interactions in the network. This method d…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning
Models for Visual Question Answering (VQA) often rely on the spurious correlations, i.e., the language priors, that appear in the biased samples of training set, which make them brittle against the out-of-distribution (O…
Contrastive LearningQuestion AnsweringVisual Question AnsweringVisual Question Answering (VQA)Learning content and context with language bias for Visual Question Answering
Visual Question Answering (VQA) is a challenging multimodal task to answer questions about an image. Many works concentrate on how to reduce language bias which makes models answer questions ignoring visual content and l…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Language bias in Visual Question Answering: A Survey and Taxonomy
Visual question answering (VQA) is a challenging task, which has attracted more and more attention in the field of computer vision and natural language processing. However, the current visual question answering has the p…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)SC-ML: Self-supervised Counterfactual Metric Learning for Debiased Visual Question Answering
Visual question answering (VQA) is a critical multimodal task in which an agent must answer questions according to the visual cue. Unfortunately, language bias is a common problem in VQA, which refers to the model genera…
counterfactualMetric LearningQuestion AnsweringVisual Question Answering+1