Chain of Reasoning for Visual Question Answering
Reasoning plays an essential role in Visual Question Answering (VQA). Multi-step and dynamic reasoning is often necessary for answering complex questions. For example, a question "What is placed next to the bus on the right of the picture?" talks about a compound object "bus on the right," which is generated by the relation <bus, on the right of, picture>. Furthermore, a new relation including this compound object <sign, next to, bus on the right> is then required to infer the answer. However, previous methods support either one-step or static reasoning, without updating relations or generating compound objects. This paper proposes a novel reasoning model for addressing these problems. A chain of reasoning (CoR) is constructed for supporting multi-step and dynamic reasoning on changed relations and objects. In detail, iteratively, the relational reasoning operations form new relations between objects, and the object refining operations generate new compound objects from relations. We achieve new state-of-the-art results on four publicly available datasets. The visualization of the chain of reasoning illustrates the progress that the CoR generates new compound objects that lead to the answer of the question step by step.
Code (0)
등록된 구현이 없습니다.
Tasks
ObjectQuestion AnsweringRelationRelational ReasoningVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Co-VQA : Answering by Interactive Sub Question Sequence
Most existing approaches to Visual Question Answering (VQA) answer questions directly, however, people usually decompose a complex question into a sequence of simple sub questions and finally obtain the answer to the ori…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Visual ReasoningCo-VQA : Answering by Interactive Sub Question Sequence
Most existing approaches to Visual Question Answering (VQA) answer questions directly, however, people usually decompose a complex question into a sequence of simple sub questions and finally obtain the answer to the ori…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Visual ReasoningReasoning over Hybrid Chain for Table-and-Text Open Domain Question Answering
Tabular and textual question answering requires systems to perform reasoning over heterogeneous information, considering table structure, and the connections among table and text. In this paper, we propose a ChAin-centri…
Open-Domain Question AnsweringQuestion AnsweringThought-For-Food: Reasoning Chain Induced Food Visual Question Answering
The immense diversity in the culture and culinary of Indian cuisines calls attention to the major shortcoming of the existing Visual Question Answering(VQA) systems which are inclined towards the foods from Western regio…
Visual Question AnsweringReinforcement LearningTowards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions
Document Visual Question Answering (DocVQA) requires vision-language models to reason not only about what information in a document is relevant to a question, but also where the answer is grounded on the page. Existing D…
Visual Question Answering