Leveraging Medical Visual Question Answering with Supporting Facts
In this working notes paper, we describe IBM Research AI (Almaden) team's participation in the ImageCLEF 2019 VQA-Med competition. The challenge consists of four question-answering tasks based on radiology images. The diversity of imaging modalities, organs and disease types combined with a small imbalanced training set made this a highly complex problem. To overcome these difficulties, we implemented a modular pipeline architecture that utilized transfer learning and multi-task learning. Our findings led to the development of a novel model called Supporting Facts Network (SFN). The main idea behind SFN is to cross-utilize information from upstream tasks to improve the accuracy on harder downstream ones. This approach significantly improved the scores achieved in the validation set (18 point improvement in F-1 score). Finally, we submitted four runs to the competition and were ranked seventh.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityMedical Visual Question AnsweringMulti-Task LearningQuestion AnsweringTransfer LearningVisual Question AnsweringVisual Question Answering (VQA)Similar Papers 제목 키워드 기반
Masked Vision and Language Pre-training with Unimodal and Multimodal Contrastive Losses for Medical Visual Question Answering
Medical visual question answering (VQA) is a challenging task that requires answering clinical questions of a given medical image, by taking consider of both visual and language information. However, due to the small sca…
Language ModelingMedical Visual Question AnsweringDynamic Memory Networks for Visual and Textual Question Answering
Neural network architectures with memory and attention mechanisms exhibit certain reasoning capabilities required for question answering. One such architecture, the dynamic memory network (DMN), obtained high accuracy on…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering
Medical Visual Question Answering (MedVQA) presents a significant opportunity to enhance diagnostic accuracy and healthcare delivery by leveraging artificial intelligence to interpret and answer questions based on medica…
BenchmarkingDiagnosticGenerative Visual Question AnsweringLanguage Modelling+5Overcoming Data Limitation in Medical Visual Question Answering
Traditional approaches for Visual Question Answering (VQA) require large amount of labeled data for training. Unfortunately, such large scale data is usually not available for medical domain. In this paper, we propose a …
DenoisingMedical Visual Question AnsweringMeta-LearningQuestion Answering+2FVQA: Fact-based Visual Question Answering
Visual Question Answering (VQA) has attracted a lot of attention in both Computer Vision and Natural Language Processing communities, not least because it offers insight into the relationships between two important sourc…
Common Sense ReasoningQuestion AnsweringTripletVisual Question Answering+1