홈 › Papers

SkinCLIP-VL: Consistency-Aware Vision-Language Learning for Multimodal Skin Cancer Diagnosis

2026-03-22 · Zhixiang Lu, Shijie Xu, Kaicheng Yan, Xuyue Cai, Chong Zhang, Yulong Li, Angelos Stefanidis, Anh Nguyen, Jionglong Su arxiv

The deployment of vision-language models (VLMs) in dermatology is hindered by the trilemma of high computational costs, extreme data scarcity, and the black-box nature of deep learning. To address these challenges, we present SkinCLIP-VL, a resource-efficient framework that adapts foundation models for trustworthy skin cancer diagnosis. Adopting a frozen perception, adaptive reasoning paradigm, we integrate a frozen CLIP encoder with a lightweight, quantized Qwen2.5-VL via low-rank adaptation (LoRA). To strictly align visual regions with clinical semantics under long-tailed distributions, we propose the Consistency-aware Focal Alignment (CFA) Loss. This objective synergizes focal re-weighting, semantic alignment, and calibration. On ISIC and Derm7pt benchmarks, SkinCLIP-VL surpasses 13B-parameter baselines by 4.3-6.2% in accuracy with 43% fewer parameters. Crucially, blinded expert evaluation and out-of-distribution testing confirm that our visually grounded rationales significantly enhance clinical trust compared to traditional saliency maps.

📄 PDF Abstract BibTeX arXiv:2603.21010

Code (0)

등록된 구현이 없습니다.

Multimodal Large Language Models for Real-Time Situated Reasoning

2026-02-02 · Giulio Antonio Abbo, Senne Lenaerts, Tony Belpaeme arxiv

In this work, we explore how multimodal large language models can support real-time context- and value-aware decision-making. To do so, we combine the GPT-4o language model with a TurtleBot 4 platform simulating a smart …

MMA-ASIA: A Multilingual and Multimodal Alignment Framework for Culturally-Grounded Evaluation

2025-10-07 · Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu 외 arxiv

Large language models (LLMs) are now used worldwide, yet their multimodal understanding and reasoning often degrade outside Western, high-resource settings. We propose MMA-ASIA, a comprehensive framework to evaluate LLMs…

Visual Question Answering

Multimodal Matching-aware Co-attention Networks with Mutual Knowledge Distillation for Fake News Detection

2022-12-12 · Linmei Hu, Ziwang Zhao, Weijian Qi, Xuemeng Song 외

Fake news often involves multimedia information such as text and image to mislead readers, proliferating and expanding its influence. Most existing fake news detection methods apply the co-attention mechanism to fuse mul…

Fake News Detection Image-text matching Knowledge Distillation Text Matching

Hierarchy-Aware Fine-Tuning of Vision-Language Models

2025-12-25 · Jiayu Li, Rajesh Gangireddy, Samet Akcay, Wei Cheng 외 arxiv

Vision-Language Models (VLMs) learn powerful multimodal representations through large-scale image-text pretraining, but adapting them to hierarchical classification is underexplored. Standard approaches treat labels as f…

PolarMem: A Training-Free Polarized Latent Graph Memory for Verifiable Vision-Language Models

2026-01-31 · Zhisheng Chen, Tingyu Wu, Zijie Zhou, Zhengwei Xie 외 arxiv

Memory is not merely a storage mechanism for intelligent systems, but a structure for organizing evidence and constraining belief. This is especially important for multimodal reasoning, where retrieved evidence must be b…

Multimodal Reasoning Semantic Similarity