Differentiable End-to-End Program Executor for Sample and Computationally Efficient VQA
We present a differentiable end-to-end program executor (DePe), which addresses Visual Question Answering (VQA) in a sample and computationally efficient manner. DePe parses the question into probabilistic programs and softly executes them to acquire the final answer. These functional programs adopt soft-logic functions to enable approximate probabilistic logic reasoning. In addition to the language, DePe also jointly learns visual object-centric representations in an end-to-end manner. We demonstrate through extensive experiments that DePe is more sample and computationally efficient than other VQA methodologies while retaining state-of-the-art performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
How to Design Sample and Computationally Efficient VQA Models
In multi-modal reasoning tasks, such as visual question answering (VQA), there have been many modeling and training paradigms tested. Previous models propose different methods for the vision and language tasks, but which…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Learning to Infer 3D Shape Programs with Differentiable Renderer
Given everyday artifacts, such as tables and chairs, humans recognize high-level regularities within them, such as the symmetries of a table, the repetition of its legs, while possessing low-level priors of their geometr…
What's Left? Concept Grounding with Logic-Enhanced Foundation Models
Recent works such as VisProg and ViperGPT have smartly composed foundation models for visual reasoning-using large language models (LLMs) to produce programs that can be executed by pre-trained vision-language models. Ho…
Visual Question Answering (VQA) Split AVisual Question Answering (VQA) Split BVisual ReasoningWhat’s Left? Concept Grounding with Logic-Enhanced Foundation Models
Recent works such as VisProg and ViperGPT have smartly composed foundation models for visual reasoning—using large language models (LLMs) to produce programs that can be executed by pre-trained vision-language models. Ho…
ARCANA: A Reflective Multi-Agent Program Synthesis Framework for ARC-AGI-2 Reasoning
We present ARCANA, a collaborative multi agent framework for solving ARC AGI 2 tasks under strict test time and hardware constraints. ARCANA decomposes each task into iterative perception, hypothesis generation, symbolic…
Program Synthesis