Compression Strategies for Efficient Multimodal LLMs in Medical Contexts
Multimodal Large Language Models (MLLMs) hold huge potential for usage in the medical domain, but their computational costs necessitate efficient compression techniques. This paper evaluates the impact of structural pruning and activation-aware quantization on a fine-tuned LLAVA model for medical applications. We propose a novel layer selection method for pruning, analyze different quantization techniques, and assess the performance trade-offs in a prune-SFT-quantize pipeline. Our proposed method enables MLLMs with 7B parameters to run within 4 GB of VRAM, reducing memory usage by 70% while achieving 4% higher model performance compared to traditional pruning and quantization techniques in the same compression ratio.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Evaluating Strategies for Synthesizing Clinical Notes for Medical Multimodal AI
Multimodal (MM) learning is emerging as a promising paradigm in biomedical artificial intelligence (AI) applications, integrating complementary modality, which highlight different aspects of patient health. The scarcity …
Cross-Modal RetrievalA Survey of Token Compression for Efficient Multimodal Large Language Models
Multimodal large language models (MLLMs) have made remarkable strides, largely driven by their ability to process increasingly long and complex contexts, such as high-resolution images, extended video sequences, and leng…
Reasoning LLMs in the Medical Domain: A Literature Survey
The emergence of advanced reasoning capabilities in Large Language Models (LLMs) marks a transformative development in healthcare applications. Beyond merely expanding functional capabilities, these reasoning mechanisms …
Reinforcement LearningInformation RetrievalPredicting Sentence Acceptability Judgments in Multimodal Contexts
Previous work has examined the capacity of deep neural networks (DNNs), particularly transformers, to predict human sentence acceptability judgments, both independently of context, and in document contexts. We consider t…
Last But Not Least: Boundary Attention CalibratiON for Multimodal KV Cache Compression
Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning, but long visual contexts enlarge the KV cache and increase decoding latency. Existing compression methods rely on observation window atte…