Parameter-Efficient Adapter Tuning for Tabular-Image Multimodal Learning
Tabular-image multimodal learning aims to improve predictive modeling by jointly using structured tabular attributes and visual data. Although pretrained encoders provide strong modality-specific representations, full fine-tuning can be computationally expensive, while keeping encoders frozen may limit task-specific adaptation. We propose the Tabular-Image Adapter (TI-Adapter), a modality-specific adapter-based fine-tuning framework for efficient multimodal adaptation. TI-Adapter freezes the pretrained tabular encoder and learns an adapter after the extracted tabular embedding, while adapting the image branch with embedding-level and bottleneck-level adapters instead of full fine-tuning. Experiments on 20 tabular-image datasets show that TI-Adapter achieves competitive or better predictive performance than full fine-tuning while using substantially fewer trainable parameters. Ablation studies further demonstrate the importance of adapter placement for balancing performance and practical efficiency.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Adapter-state Sharing CLIP for Parameter-efficient Multimodal Sarcasm Detection
The growing prevalence of multimodal image-text sarcasm on social media poses challenges for opinion mining systems. Existing approaches rely on full fine-tuning of large models, making them unsuitable to adapt under res…
parameter-efficient fine-tuningSarcasm DetectionOpinion MiningRSAdapter: Adapting Multimodal Models for Remote Sensing Visual Question Answering
In recent years, with the rapid advancement of transformer models, transformer-based multimodal architectures have found wide application in various downstream tasks, including but not limited to Image Captioning, Visual…
Image CaptioningQuestion AnsweringText GenerationVisual Question Answering+1CROME: Cross-Modal Adapters for Efficient Multimodal LLM
Multimodal Large Language Models (MLLMs) demonstrate remarkable image-language capabilities, but their widespread use faces challenges in cost-effective training and adaptation. Existing approaches often necessitate expe…
Instruction FollowingLanguage ModelingLanguage ModellingQuestion Answering+1Pilot: Building the Federated Multimodal Instruction Tuning Framework
In this paper, we explore a novel federated multimodal instruction tuning task(FedMIT), which is significant for collaboratively fine-tuning MLLMs on different types of multimodal instruction data on distributed devices.…
General KnowledgeQ-Adapter: Visual Query Adapter for Extracting Textually-related Features in Video Captioning
Recent advances in video captioning are driven by large-scale pretrained models, which follow the standard "pre-training followed by fine-tuning" paradigm, where the full model is fine-tuned for downstream tasks. Althoug…
parameter-efficient fine-tuningVideo Captioning