Global Prompt Cell: A Portable Control Module for Effective Prompt Tuning
As a novel approach to tuning pre-trained models, prompt tuning involves freezing the parameters in downstream tasks while inserting trainable embeddings into inputs in the first layer. However, previous methods have mainly focused on the initialization of prompt embeddings. The strategy of training and utilizing prompt embeddings in a reasonable way has become a limiting factor in the effectiveness of prompt tuning. To address this issue, we introduce the Global Prompt Cell (GPC), a portable control module for prompt tuning that selectively preserves prompt information across all encoder layers. Our experimental results demonstrate a 5.8% improvement on SuperGLUE datasets compared to vanilla prompt tuning.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
The Mirror Design Pattern: Strict Data Geometry over Model Scale for Prompt Injection Detection
Prompt injection defenses are often framed as semantic understanding problems and delegated to increasingly large neural detectors. For the first screening layer, however, the requirements are different: the detector run…
Portable-CELLxGENE: standalone executables of CELLxGENE for easy installation
Biologists who want to analyse their single-cell transcriptomics dataset must install and use specialist software via the command line. This is often impractical for non-bioinformaticians. Whilst the popular CELLxGENE so…
Dynamic Cell Modeling of Li-Ion Polymer Batteries for Precise SOC Estimation in Power-Needy Autonomous Electric Vehicles
This paper presents findings on dynamic cell modeling for state-of-charge (SOC) estimation in an autonomous electric vehicle (AEV). The studied cells are Lithium-Ion Polymer-based with a nominal capacity of around 8Ah, o…
QuantizationCell Balancing for the Transportation Sector: Techniques, Challenges, and Future Research Directions
Efficient and reliable energy systems are key to progress of society. High performance batteries are essential for widely used technologies like Electric Vehicles (EVs) and portable electronics. Additionally, an effectiv…
ManagementSignal or Noise? A Benchmark Study of Agent Skills in Web Development
Agent Skills are reusable procedural modules that are increasingly injected into coding-agent sessions to encode framework conventions, anti-patterns, and reusable tools. However, because each injected Skill expands the …