Hyper-dimensional computing for a visual question-answering system that is trainable end-to-end
In this work we propose a system for visual question answering. Our architecture is composed of two parts, the first part creates the logical knowledge base given the image. The second part evaluates questions against the knowledge base. Differently from previous work, the knowledge base is represented using hyper-dimensional computing. This choice has the advantage that all the operations in the system, namely creating the knowledge base and evaluating the questions against it, are differentiable, thereby making the system easily trainable in an end-to-end fashion.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Hypergraph Transformer: Weakly-supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering
Knowledge-based visual question answering (QA) aims to answer a question which requires visually-grounded external knowledge beyond image content itself. Answering complex questions that require multi-hop reasoning under…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)CHOPT : Automated Hyperparameter Optimization Framework for Cloud-Based Machine Learning Platforms
Many hyperparameter optimization (HyperOpt) methods assume restricted computing resources and mainly focus on enhancing performance. Here we propose a novel cloud-based HyperOpt (CHOPT) framework which can efficiently ut…
BIG-bench Machine LearningHyperparameter OptimizationQuestion AnsweringVSA4VQA: Scaling a Vector Symbolic Architecture to Visual Question Answering on Natural Images
While Vector Symbolic Architectures (VSAs) are promising for modelling spatial cognition, their application is currently limited to artificially generated images and simple spatial queries. We propose VSA4VQA - a novel 4…
AttributeLanguage ModelingLanguage ModellingQuestion Answering+2SHMamba: Structured Hyperbolic State Space Model for Audio-Visual Question Answering
The Audio-Visual Question Answering (AVQA) task holds significant potential for applications. Compared to traditional unimodal approaches, the multi-modal input of AVQA makes feature extraction and fusion processes more …
Audio-visual Question AnsweringAudio-Visual Question Answering (AVQA)Question AnsweringState Space Models+1Hypercube-RAG: Hypercube-Based Retrieval-Augmented Generation for In-domain Scientific Question-Answering
Large language models (LLMs) often need to incorporate external knowledge to solve theme-specific problems. Retrieval-augmented generation (RAG), which empowers LLMs to generate more qualified responses with retrieved ex…
Question AnsweringRAGRetrievalRetrieval-augmented Generation+2