LEAF-QA: Locate, Encode & Attend for Figure Question Answering
We introduce LEAF-QA, a comprehensive dataset of $250,000$ densely annotated figures/charts, constructed from real-world open data sources, along with ~2 million question-answer (QA) pairs querying the structure and semantics of these charts. LEAF-QA highlights the problem of multimodal QA, which is notably different from conventional visual QA (VQA), and has recently gained interest in the community. Furthermore, LEAF-QA is significantly more complex than previous attempts at chart QA, viz. FigureQA and DVQA, which present only limited variations in chart data. LEAF-QA being constructed from real-world sources, requires a novel architecture to enable question answering. To this end, LEAF-Net, a deep architecture involving chart element localization, question and answer encoding in terms of chart elements, and an attention network is proposed. Different experiments are conducted to demonstrate the challenges of QA on LEAF-QA. The proposed architecture, LEAF-Net also considerably advances the current state-of-the-art on FigureQA and DVQA.
Code (0)
등록된 구현이 없습니다.
Tasks
Chart Question AnsweringQuestion AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Forecasting Transformative AI: An Expert Survey
Transformative AI technologies have the potential to reshape critical aspects of society in the near future. However, in order to properly prepare policy initiatives for the arrival of such technologies accurate forecast…
SurveyLeaf: Multiple-Choice Question Generation
Testing with quiz questions has proven to be an effective way to assess and improve the educational process. However, manually creating quizzes is tedious and time-consuming. To address this challenge, we present Leaf, a…
Multiple-choiceQuestion AnsweringQuestion GenerationQuestion-GenerationSemantic Segmentation for Compound figures
Scientific literature contains large volumes of unstructured data,with over 30\% of figures constructed as a combination of multiple images, these compound figures cannot be analyzed directly with existing information re…
Information RetrievalRetrievalSegmentationSemantic SegmentationPACSET (Packed Serialized Trees): Reducing Inference Latency for Tree Ensemble Deployment
We present methods to serialize and deserialize tree ensembles that optimize inference latency when models are not already loaded into memory. This arises whenever models are larger than memory, but also systematically w…
A Better Way to Attend: Attention with Trees for Video Question Answering
We propose a new attention model for video question answering. The main idea of the attention models is to locate on the most informative parts of the visual data. The attention mechanisms are quite popular these days. H…
Question AnsweringVideo Question Answering