Think Visually: Question Answering through Virtual Imagery
In this paper, we study the problem of geometric reasoning in the context of question-answering. We introduce Dynamic Spatial Memory Network (DSMN), a new deep network architecture designed for answering questions that admit latent visual representations. DSMN learns to generate and reason over such representations. Further, we propose two synthetic benchmarks, FloorPlanQA and ShapeIntersection, to evaluate the geometric reasoning capability of QA systems. Experimental results validate the effectiveness of our proposed DSMN for visual thinking tasks.
Code (1)
Tasks
Question AnsweringVisual Commonsense ReasoningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Ontology-based question answering over corporate structured data
Ontology-based approach to the Natural Language Understanding (NLU) processing allows to improve questions answering quality in dialogue systems. We describe our NLU engine architecture and evaluate its implementation. T…
Natural Language UnderstandingQuestion AnsweringThinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Humans possess the visual-spatial intelligence to remember spaces from sequential visual observations. However, can Multimodal Large Language Models (MLLMs) trained on million-scale video datasets also ``think in space''…
Question AnsweringSpatial ReasoningLearning to Summarize and Answer Questions about a Virtual Robot's Past Actions
When robots perform long action sequences, users will want to easily and reliably find out what they have done. We therefore demonstrate the task of learning to summarize and answer questions about a robot agent's past a…
Language ModelingLanguage ModellingLarge Language ModelQuestion AnsweringSee and Think: Embodied Agent in Virtual Environment
Large language models (LLMs) have achieved impressive pro-gress on several open-world tasks. Recently, using LLMs to build embodied agents has been a hotspot. This paper proposes STEVE, a comprehensive and visionary embo…
MinecraftQuestion AnsweringRetrievalUniOQA: A Unified Framework for Knowledge Graph Question Answering with Large Language Models
OwnThink stands as the most extensive Chinese open-domain knowledge graph introduced in recent times. Despite prior attempts in question answering over OwnThink (OQA), existing studies have faced limitations in model rep…
Graph Question AnsweringQuestion AnsweringRAGRetrieval-augmented Generation+1