Context-Aware Prompt: Customize A Unique Prompt For Each Input

2022-01-16 · ACL ARR January 2022 1 · Anonymous

After the proposal of BERT, pre-trained language models have become the dominant approach for solving many NLP tasks. Typically, a linear classifier is added to the head of the model for fine-tuning to fit downstream tasks, while a more recent approach, also known as prompt-based learning or prompt-learning, using prompts to perform various downstream tasks, is considered to be able to uncover the potential of the language model.Prior study, however, attempted to find a universal prompt for a certain task across all samples. Therefore, we propose a novel method, Context-Aware Prompt (CAP), which provides a unique continuous prompt for each sample input by combining contextual information to further investigate the potential capabilities of the language models. On the SuperGlue benchmark, our method outperforms multiple models with vanilla fine-tuning. Furthermore, we extend the use of prompts to include Replaced Token Detection (RTD) type prompts, allowing models like ELECTRA and DeBERTaV3 that employ RTD as a training objective to use prompts for downstream tasks.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Prompt Learning

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-Head Attention 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Adam 설명 없음

Attention Dropout Attention Dropout is a type of dropout used in attention-based architectures, where elements are randomly dropped out of the…

ELECTRA 설명 없음

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent

2024-07-23 · Huiyu Xu, Wenhui Zhang, Zhibo Wang, Feng Xiao 외

Recently, advanced Large Language Models (LLMs) such as GPT-4 have been integrated into many real-world applications like Code Copilot. These applications have significantly expanded the attack surface of LLMs, exposing …

Red Teaming

In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulation

2025-05-26 · Yu Xu, Fan Tang, You Wu, Lin Gao 외

Recent advances in diffusion models have enhanced multimodal-guided visual generation, enabling customized subject insertion that seamlessly "brushes" user-specified objects into a given image guided by textual prompts. …

In-Context Learning

PromptDLA: A Domain-aware Prompt Document Layout Analysis Framework with Descriptive Knowledge as a Cue

2026-03-10 · Zirui Zhang, Yaping Zhang, Lu Xiang, Yang Zhao 외 arxiv

Document Layout Analysis (DLA) is crucial for document artificial intelligence and has recently received increasing attention, resulting in an influx of large-scale public DLA datasets. Existing work often combines data …

Document Layout Analysis

Membership Inference Attacks on In-Context Examples in LLM-based Recommender Systems

2025-08-26 · Jiajie He, Min-Chun Chen, Xintong Chen, Xinyang Fang 외 arxiv

Large language models (LLMs) based recommender systems (RecSys) can adapt flexibly across different domains. It uses in-context learning (ICL), i.e., prompts, including sensitive historical user-specific item interaction…

RelationBooth: Towards Relation-Aware Customized Object Generation

2024-10-30 · Qingyu Shi, Lu Qi, Jianzong Wu, Jinbin Bai 외

Customized image generation is crucial for delivering personalized content based on user-provided image prompts, aligning large-scale text-to-image diffusion models with individual needs. However, existing models often o…

Image Generation Object Relation