CrossPT: Exploring Cross-Task Transferability through Multi-Task Prompt Tuning
Prompt tuning offers a parameter-efficient way to adapt large pre-trained language models to new tasks, but most existing approaches are designed for single-task settings, failing to share knowledge across related tasks. We propose Cross-task Prompt Tuning (CrossPT), a modular framework for multi-task prompt tuning that enables controlled knowledge transfer while maintaining task-specific specialization. CrossPT decomposes each target prompt into shared, pre-trained source prompts and task-specific private prompts, combined via a learned attention mechanism. To support robust transfer, we systematically investigate key design factors including prompt initialization, balancing shared and private prompts, number of source prompts, learning rates, task prefixes, and label semantics. Empirical results on GLUE and related benchmarks show that CrossPT achieves higher accuracy and robustness compared to traditional prompt tuning and related methods, particularly in low-resource scenarios, while maintaining strong parameter efficiency.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Exploring Robust Intrusion Detection: A Benchmark Study of Feature Transferability in IoT Botnet Attack Detection
Cross-domain intrusion detection remains a critical challenge due to significant variability in network traffic characteristics and feature distributions across environments. This study evaluates the transferability of t…
Feature EngineeringIntrusion DetectionFeature ImportanceExploring and Predicting Transferability across NLP Tasks
Recent advances in NLP demonstrate the effectiveness of training large-scale language models and transferring them to downstream tasks. Can fine-tuning these models on tasks other than language modeling further improve p…
Language ModelingLanguage ModellingPart-Of-Speech TaggingQuestion Answering+3Exploring the BERT Cross-Lingual Transferability: a Case Study in Reading Comprehension
Multilingual BERT has been shown to generalize well in a zero-shot cross-lingual setting. This generalization was measured on POS and NER tasks. We explore the multilingual BERT cross-language transferability on the read…
NERPOSQuestion AnsweringReading ComprehensionBoosting Adversarial Transferability for Skeleton-based Action Recognition via Exploring the Model Posterior Space
Skeletal motion plays a pivotal role in human activity recognition (HAR). Recently, attack methods have been proposed to identify the universal vulnerability of skeleton-based HAR(S-HAR). However, the research of adversa…
Action RecognitionActivity RecognitionHuman Activity RecognitionSkeleton Based Action RecognitionExploring Transferability of Self-Supervised Learning by Task Conflict Calibration
In this paper, we explore the transferability of SSL by addressing two central questions: (i) what is the representation transferability of SSL, and (ii) how can we effectively model this transferability? Transferability…
Self-Supervised Learning