Increasing Textual Context Size Boosts Medical Image-Text Matching
This short technical report demonstrates a simple technique that yields state of the art results in medical image-text matching tasks. We analyze the use of OpenAI's CLIP, a general image-text matching model, and observe that CLIP's limited textual input size has negative impact on downstream performance in the medical domain where encoding longer textual contexts is often required. We thus train and release ClipMD, which is trained with a simple sliding window technique to encode textual captions. ClipMD was tested on two medical image-text datasets and compared with other image-text matching models. The results show that ClipMD outperforms other models on both datasets by a large margin. We make our code and pretrained model publicly available.
Code (1)
Tasks
Image-text matchingText MatchingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Instruction Tuning and CoT Prompting for Contextual Medical QA with LLMs
Large language models (LLMs) have shown great potential in medical question answering (MedQA), yet adapting them to biomedical reasoning remains challenging due to domain-specific complexity and limited supervision. In t…
Medical Question AnsweringMedQAMultiple-choicePrompt Engineering+1DeCode: Decoupling Content and Delivery for Medical QA
Large language models (LLMs) exhibit strong medical knowledge and can generate factually accurate responses. However, existing models often fail to account for individual patient contexts, producing answers that are clin…
Question AnsweringReconstruction Probing
We propose reconstruction probing, a new analysis method for contextualized representations based on reconstruction probabilities in masked language models (MLMs). This method relies on comparing the reconstruction proba…
A Strictly Bounded Deep Network for Unpaired Cyclic Translation of Medical Images
Medical image translation is an ill-posed problem. Unlike existing paired unbounded unidirectional translation networks, in this paper, we consider unpaired medical images and provide a strictly bounded network that yiel…
Dictionary LearningGenerative Adversarial NetworkTranslation3MDBench: Medical Multimodal Multi-agent Dialogue Benchmark
Though Large Vision-Language Models (LVLMs) are being actively explored in medicine, their ability to conduct telemedicine consultations combining accurate diagnosis with professional dialogue remains underexplored. In t…
DiagnosticMultimodal Reasoning