Visual Question Generation as Dual Task of Visual Question Answering

2017-09-21 · CVPR 2018 6 · Yikang Li, Nan Duan, Bolei Zhou, Xiao Chu, Wanli Ouyang, Xiaogang Wang

Recently visual question answering (VQA) and visual question generation (VQG) are two trending topics in the computer vision, which have been explored separately. In this work, we propose an end-to-end unified framework, the Invertible Question Answering Network (iQAN), to leverage the complementary relations between questions and answers in images by jointly training the model on VQA and VQG tasks. Corresponding parameter sharing scheme and regular terms are proposed as constraints to explicitly leverage Q,A's dependencies to guide the training process. After training, iQAN can take either question or answer as input, then output the counterpart. Evaluated on the large-scale visual question answering datasets CLEVR and VQA2, our iQAN improves the VQA accuracy over the baselines. We also show the dual learning framework of iQAN can be generalized to other VQA architectures and consistently improve the results over both the VQA and VQG tasks.

📄 PDF Abstract BibTeX arXiv:1709.07192

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering Question Generation Question-Generation Visual Question Answering Visual Question Answering (VQA)

Generating Rationales in Visual Question Answering

2020-04-04 · Hammad A. Ayyubi, Md. Mehrab Tanjim, Julian J. McAuley, Garrison W. Cottrell

Despite recent advances in Visual QuestionAnswering (VQA), it remains a challenge todetermine how much success can be attributedto sound reasoning and comprehension ability.We seek to investigate this question by propos-…

Question Answering Visual Question Answering Visual Question Answering (VQA)

Tell Me the Evidence? Dual Visual-Linguistic Interaction for Answer Grounding

2022-06-21 · Junwen Pan, Guanlin Chen, Yi Liu, Jiexiang Wang 외

Answer grounding aims to reveal the visual evidence for visual question answering (VQA), which entails highlighting relevant positions in the image when answering questions about images. Previous attempts typically tackl…

Decoder Question Answering Visual Grounding Visual Question Answering+1

CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering

2025-03-01 · CVPR 2025 1 · Tianyu Huai, Jie zhou, Xingjiao Wu, Qin Chen 외

Multimodal large language models (MLLMs) have garnered widespread attention from researchers due to their remarkable understanding and generation capabilities in visual language tasks (e.g., visual question answering). H…

Continual Learning Language Modeling Language Modelling Large Language Model+5

ConVQG: Contrastive Visual Question Generation with Multimodal Guidance

2024-02-20 · Li Mi, Syrielle Montariol, Javiera Castillo-Navarro, Xianjie Dai 외

Asking questions about visual environments is a crucial way for intelligent agents to understand rich multi-faceted scenes, raising the importance of Visual Question Generation (VQG) systems. Apart from being grounded to…

Question Generation Question-Generation

BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering

2026-04-24 · Jinghong Chen, Jingbiao Mei, Guangyu Yang, Bill Byrne arxiv

A common approach to question answering with retrieval-augmented generation (RAG) is to concatenate documents into a single context and pass it to a language model to generate an answer. While simple, this strategy can o…

Visual Question Answering