Adapter Merging with Centroid Prototype Mapping for Scalable Class-Incremental Learning

2024-12-24 · CVPR 2025 1 · Takuma Fukuda, Hiroshi Kera, Kazuhiko Kawamoto

We propose Adapter Merging with Centroid Prototype Mapping (ACMap), an exemplar-free framework for class-incremental learning (CIL) that addresses both catastrophic forgetting and scalability. While existing methods trade-off between inference time and accuracy, ACMap consolidates task-specific adapters into a single adapter, ensuring constant inference time across tasks without compromising accuracy. The framework employs adapter merging to build a shared subspace that aligns task representations and mitigates forgetting, while centroid prototype mapping maintains high accuracy through consistent adaptation in the shared subspace. To further improve scalability, an early stopping strategy limits adapter merging as tasks increase. Extensive experiments on five benchmark datasets demonstrate that ACMap matches state-of-the-art accuracy while maintaining inference time comparable to the fastest existing methods. The code is available at https://github.com/tf63/ACMap

📄 PDF Abstract BibTeX arXiv:2412.18219

Code (1)

tf63/acmap 공식 구현 pytorch

Tasks

class-incremental learning Class Incremental LearningExemplar-FreeIncremental Learning

Methods 이 논문이 사용한 방법론

Early Stopping Early Stopping is a regularization technique for deep neural networks that stops training when parameter updates no longer begin to yield improves on a validation set. In…

Adapter 설명 없음

Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts

2025-07-09 · Samin Yeasar Arnob, Zhan Su, Minseon Kim, Oleksiy Ostapenko 외

Merging parameter-efficient task experts has recently gained growing attention as a way to build modular architectures that can be rapidly adapted on the fly for specific downstream tasks, without requiring additional fi…

Pin Once, Swap Light: Subspace-Aligned Centroid-Residual Training for Efficient Ultra-LoRA Serving

2026-08-04 · Xiang Li, Pengcheng Wang, Huazheng Wang, Saurabh Bagchi arxiv

Modern multi-tenant Low-Rank Adapters (LoRAs) serving systems concurrently host tens to hundreds of LoRA adapters. Though powerful, this introduces a critical system dilemma between serving efficiency and task performanc…

ARIADNE: Agnostic Routing for Inference-time Adapter DyNamic sElection

2026-06-17 · Enrico Cassano, Michał Brzozowski, Zuzanna Dubanowska, Paolo Mandica 외 arxiv

The increasing deployment of parameter-efficient fine-tuning (PEFT) has led to model ecosystems in which a single backbone is paired with many task-specialized adapters. In this setting, inference-time queries often arri…

parameter-efficient fine-tuning

ConMoE: Expert-Pool Consolidation via Prototype Reassignment for MoE Compression

2026-05-28 · Yilun Yao, Jiaming Pan, Elsie Dai, Peizhuang Cong 외 arxiv

Mixture-of-Experts (MoE) language models reduce per-token computation but still require storing and serving all experts, making deployment memory-intensive. Existing post-training compression methods mainly shrink this c…

Task-Aware LoRA Adapter Composition via Similarity Retrieval in Vector Databases

2026-02-01 · Riya Adsul, Balachandra Devarangadi Sunil, Isha Nalawade, Sudharshan Govindan arxiv

Parameter efficient fine tuning methods like LoRA have enabled task specific adaptation of large language models, but efficiently composing multiple specialized adapters for unseen tasks remains challenging. We present a…

Natural Language Inference Zero-shot Generalization Sentiment Analysis Question Answering