What and Where to Adapt: Structure-Semantics Co-Tuning for Machine Vision Compression via Synergistic Adapters

2026-04-11 · Shaobo Liu, Haobo Xiong, Kai Liu, Yuna Lin arxiv

Parameter-efficient fine-tuning of pre-trained codecs is a promising direction in image compression for human and machine vision. While most existing works have primarily focused on tuning the feature structure within the encoder-decoder backbones, the adaptation of the statistical semantics within the entropy model has received limited attention despite its function of predicting the probability distribution of latent features. Our analysis reveals that naive adapter insertion into the entropy model can lead to suboptimal outcomes, underscoring that the effectiveness of adapter-based tuning depends critically on the coordination between adapter type and placement across the compression pipeline. Therefore, we introduce Structure-Semantics Co-Tuning (S2-CoT), a novel framework that achieves this coordination via two specialized, synergistic adapters: the Structural Fidelity Adapter (SFA) and the Semantic Context Adapter (SCA). SFA is integrated into the encoder-decoder to preserve high-fidelity representations by dynamically fusing spatial and frequency information; meanwhile, the SCA adapts the entropy model to align with SFA-tuned features by refining the channel context for more efficient statistical coding. Through joint optimization, S2-CoT turns potential performance degradation into synergistic gains, achieving state-of-the-art results across four diverse base codecs with only a small fraction of trainable parameters, closely matching full fine-tuning performance. Code is available at https://github.com/Brock-bit4/S2-CoT.

📄 PDF Abstract BibTeX arXiv:2604.10017

Code (0)

등록된 구현이 없습니다.

Tasks

parameter-efficient fine-tuning Image Compression

GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph

2023-09-24 · NeurIPS 2023 11 · Xin Li, Dongze Lian, Zhihe Lu, Jiawang Bai 외

Adapter-style efficient transfer learning (ETL) has shown excellent performance in the tuning of vision-language models (VLMs) under the low-data regime, where only a few additional parameters are introduced to excavate …

Transfer Learning

What DINO saw: ALiBi positional encoding reduces positional bias in Vision Transformers

2026-03-17 · Moritz Pawlowsky, Antonis Vamvakeros, Alexander Weiss, Anja Bielefeld 외 arxiv

Vision transformers (ViTs) - especially feature foundation models like DINOv2 - learn rich representations useful for many downstream tasks. However, architectural choices (such as positional encoding) can lead to these …

Structure-aware Prompt Adaptation from Seen to Unseen for Open-Vocabulary Compositional Zero-Shot Learning

2026-03-04 · Yihang Duan, Jiong Wang, Pengpeng Zeng, Ji Zhang 외 arxiv

The goal of Open-Vocabulary Compositional Zero-Shot Learning (OV-CZSL) is to recognize attribute-object compositions in the open-vocabulary setting, where compositions of both seen and unseen attributes and objects are e…

Compositional Zero-Shot Learning

ProtoAda: Prototype-Guided Adaptive Adapter Expansion and Geometric Consolidation for Multimodal Continual Instruction Tuning

2026-06-01 · Yu-Cheng Shi, Zhen-Hao Xie, Jun-Tao Tang, Da-Wei Zhou arxiv

Multimodal Large Language Models (MLLMs) achieve strong performance through instruction tuning, but real-world deployment requires them to continually acquire new vision-language capabilities, making Multimodal Continual…

Exploring Strategies for Generalizable Commonsense Reasoning with Pre-trained Models

2021-09-07 · EMNLP 2021 11 · Kaixin Ma, Filip Ilievski, Jonathan Francis, Satoru Ozaki 외

Commonsense reasoning benchmarks have been largely solved by fine-tuning language models. The downside is that fine-tuning may cause models to overfit to task-specific data and thereby forget their knowledge gained durin…