Privacy-Preserving Retrieval-Augmented Generation with Differential Privacy

2024-12-06 · Tatsuki Koga, Ruihan Wu, Kamalika Chaudhuri

With the recent remarkable advancement of large language models (LLMs), there has been a growing interest in utilizing them in the domains with highly sensitive data that lies outside their training data. For this purpose, retrieval-augmented generation (RAG) is particularly effective -- it assists LLMs by directly providing relevant information from the external knowledge sources. However, without extra privacy safeguards, RAG outputs risk leaking sensitive information from the external data source. In this work, we explore RAG under differential privacy (DP), a formal guarantee of data privacy. The main challenge with differentially private RAG is how to generate long accurate answers within a moderate privacy budget. We address this by proposing an algorithm that smartly spends privacy budget only for the tokens that require the sensitive information and uses the non-private LLM for other tokens. Our extensive empirical evaluations reveal that our algorithm outperforms the non-RAG baseline under a reasonable privacy budget of $\epsilon\approx 10$ across different models and datasets.

📄 PDF Abstract BibTeX arXiv:2412.04697

Code (0)

등록된 구현이 없습니다.

Tasks

Privacy PreservingRAG RetrievalRetrieval-augmented Generation

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Multi-Head Attention 설명 없음

Weight Decay 설명 없음

WordPiece 설명 없음

Differentially Private Synthetic Text Generation for Retrieval-Augmented Generation (RAG)

2025-10-08 · Junki Mori, Kazuya Kakizaki, Taiki Miyagawa, Jun Sakuma arxiv

Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by grounding them in external knowledge. However, its application in sensitive domains is limited by privacy risks. Existing private RAG methods …

Text Generation

RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models

2025-02-18 · Tanqiu Jiang, Changjiang Li, Fenglong Ma, Ting Wang

Differentially private diffusion models (DPDMs) harness the remarkable generative capabilities of diffusion models while enforcing differential privacy (DP) for sensitive data. However, existing DPDM training approaches …

Privacy Preserving RAG Retrieval Retrieval-augmented Generation

Differentially Private Retrieval-Augmented Generation

2026-02-16 · Tingting Tang, James Flemings, Yongqin Wang, Murali Annavaram arxiv

Retrieval-augmented generation (RAG) is a widely used framework for reducing hallucinations in large language models (LLMs) on domain-specific tasks by retrieving relevant documents from a database to support accurate re…

Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generation

2025-08-05 · Haoran Wang, Xiongxiao Xu, Baixiang Huang, Kai Shu arxiv

Retrieval-Augmented Generation (RAG) enhances the factual accuracy of large language models (LLMs) by conditioning outputs on external knowledge sources. However, when retrieval involves private or sensitive data, RAG sy…

Noise-Augmented Privacy-Preserving Empirical Risk Minimization with Dual-purpose Regularizer and Privacy Budget Retrieval and Recycling

2021-10-16 · Yinan Li, Fang Liu

We propose Noise-Augmented Privacy-Preserving Empirical Risk Minimization (NAPP-ERM) that solves ERM with differential privacy guarantees. Existing privacy-preserving ERM approaches may be subject to over-regularization …

Privacy Preserving Retrieval Variable Selection