WangLab at MEDIQA-M3G 2024: Multimodal Medical Answer Generation using Large Language Models
This paper outlines our submission to the MEDIQA2024 Multilingual and Multimodal Medical Answer Generation (M3G) shared task. We report results for two standalone solutions under the English category of the task, the first involving two consecutive API calls to the Claude 3 Opus API and the second involving training an image-disease label joint embedding in the style of CLIP for image classification. These two solutions scored 1st and 2nd place respectively on the competition leaderboard, substantially outperforming the next best solution. Additionally, we discuss insights gained from post-competition experiments. While the performance of these two solutions have significant room for improvement due to the difficulty of the shared task and the challenging nature of medical visual question answering in general, we identify the multi-stage LLM approach and the CLIP image classification approach as promising avenues for further investigation.
Code (0)
등록된 구현이 없습니다.
Tasks
Answer Generationimage-classificationImage ClassificationMedical Visual Question AnsweringQuestion AnsweringVisual Question AnsweringMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MediFact at MEDIQA-M3G 2024: Medical Question Answering in Dermatology with Multimodal Learning
The MEDIQA-M3G 2024 challenge necessitates novel solutions for Multilingual & Multimodal Medical Answer Generation in dermatology (wai Yim et al., 2024a). This paper addresses the limitations of traditional methods by pr…
Answer GenerationMedical Question AnsweringQuestion AnsweringWeakly-supervised LearningWangLab at MEDIQA-CORR 2024: Optimized LLM-based Programs for Medical Error Detection and Correction
Medical errors in clinical text pose significant risks to patient safety. The MEDIQA-CORR 2024 shared task focuses on detecting and correcting these errors across three subtasks: identifying the presence of an error, ext…
DiversityLanguage ModelingLanguage ModellingLarge Language Model+3WangLab at MEDIQA-Chat 2023: Clinical Note Generation from Doctor-Patient Conversations using Large Language Models
This paper describes our submission to the MEDIQA-Chat 2023 shared task for automatic clinical note generation from doctor-patient conversations. We report results for two approaches: the first fine-tunes a pre-trained l…
In-Context LearningLanguage ModelingLanguage ModellingLarge Language ModelMediQAl: A French Medical Question Answering Dataset for Knowledge and Reasoning Evaluation
This work introduces MediQAl, a French medical question answering dataset designed to evaluate the capabilities of language models in factual medical recall and reasoning over real-world clinical scenarios. MediQAl conta…
Question AnsweringUCSD-Adobe at MEDIQA 2021: Transfer Learning and Answer Sentence Selection for Medical Summarization
In this paper, we describe our approach to question summarization and multi-answer summarization in the context of the 2021 MEDIQA shared task (Ben Abacha et al., 2021). We propose two kinds of transfer learning for the …
Abstractive Text SummarizationDecoderExtractive SummarizationSentence+1