How Private is Your Attention? Bridging Privacy with In-Context Learning
In-context learning (ICL)-the ability of transformer-based models to perform new tasks from examples provided at inference time-has emerged as a hallmark of modern language models. While recent works have investigated the mechanisms underlying ICL, its feasibility under formal privacy constraints remains largely unexplored. In this paper, we propose a differentially private pretraining algorithm for linear attention heads and present the first theoretical analysis of the privacy-accuracy trade-off for ICL in linear regression. Our results characterize the fundamental tension between optimization and privacy-induced noise, formally capturing behaviors observed in private training via iterative methods. Additionally, we show that our method is robust to adversarial perturbations of training prompts, unlike standard ridge regression. All theoretical findings are supported by extensive simulations across diverse settings.
Code (0)
등록된 구현이 없습니다.
Tasks
In-Context LearningregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer
Large Language Models (LLMs) have emerged as dominant tools for various tasks, particularly when tailored for a specific target by prompt tuning. Nevertheless, concerns surrounding data privacy present obstacles due to t…
In-Context LearningLanguage ModelingLanguage ModellingLarge Language Model+2What Your Posts Reveal: A Benchmark and Agentic Framework for User-Level Privacy Leakage on Social Media
Public social media posts can reveal private information through weak cues scattered across text, images, or metadata. Such leakage is often cumulative and cross-post: cues that appear harmless in isolation may jointly e…
How Private Is Your RL Policy? An Inverse RL Based Analysis Framework
Reinforcement Learning (RL) enables agents to learn how to perform various tasks from scratch. In domains like autonomous driving, recommendation systems, and more, optimal RL policies learned could cause a privacy breac…
Adversarial AttackAutonomous DrivingPrivacy PreservingRecommendation Systems+2Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by retrieving documents from an external corpus at inference time. When this corpus contains sensitive information, however, unprotected RAG syst…
Guarding Your Conversations: Privacy Gatekeepers for Secure Interactions with Cloud-Based AI Models
The interactive nature of Large Language Models (LLMs), which closely track user data and context, has prompted users to share personal and private information in unprecedented ways. Even when users opt out of allowing t…