paper-with-me

Papers

SD-RAG: A Prompt-Injection-Resilient Framework for Selective Disclosure in Retrieval-Augmented Generation

2026-01-16 · Aiman Al Masoud, Marco Arazzi, Antonino Nocera arxiv

Retrieval-Augmented Generation (RAG) has attracted significant attention due to its ability to combine the generative capabilities of Large Language Models (LLMs) with knowledge obtained through efficient retrieval mechanisms over large-scale data collections. Currently, the majority of existing approaches overlook the risks associated with exposing sensitive or access-controlled information directly to the generation model. Only a few approaches propose techniques to instruct the generative model to refrain from disclosing sensitive information; however, recent studies have also demonstrated that LLMs remain vulnerable to prompt injection attacks that can override intended behavioral constraints. For these reasons, we propose a novel approach to Selective Disclosure in Retrieval-Augmented Generation, called SD-RAG, which decouples the enforcement of security and privacy constraints from the generation process itself. Rather than relying on prompt-level safeguards, SD-RAG applies sanitization and disclosure controls during the retrieval phase, prior to augmenting the language model's input. Moreover, we introduce a semantic mechanism to allow the ingestion of human-readable dynamic security and privacy constraints together with an optimized graph-based data model that supports fine-grained, policy-aware retrieval. Our experimental evaluation demonstrates the superiority of SD-RAG over baseline existing approaches, achieving up to a $58\%$ improvement in the privacy score, while also showing a strong resilience to prompt injection attacks targeting the generative model.

📄 PDF Abstract BibTeX arXiv:2601.11199

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Jatmo: Prompt Injection Defense by Task-Specific Finetuning

2023-12-29 · Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen 외

Large Language Models (LLMs) are attracting significant research attention due to their instruction-following abilities, allowing users and developers to leverage LLMs for a variety of tasks. However, LLMs are vulnerable…

Instruction Following

Portable Agent Memory: A Protocol for Cryptographically-Verified Memory Transfer Across Heterogeneous AI Agents

2026-05-10 · Santhosh Kumar Ravindran arxiv

We present Portable Agent Memory, an open protocol and reference implementation for transferring persistent memory state across heterogeneous AI agents. Modern AI agents accumulate rich context -- episodic events,semanti…

SPIN: Self-Supervised Prompt INjection

2024-10-17 · Leon Zhou, Junfeng Yang, Chengzhi Mao

Large Language Models (LLMs) are increasingly used in a variety of important applications, yet their safety and reliability remain as major concerns. Various adversarial and jailbreak attacks have been proposed to bypass…

Safety Alignment

Selective Disclosure Watermarking for Large Language Models

2026-07-06 · Xuyang Chen, Xiang Li, Yangxinyu Xie, Qi Long arxiv

Watermarking methods embed imperceptible and verifiable signals into text generated by large language models (LLMs). Existing approaches include zero-bit schemes for distinguishing synthetic text from human writing and m…

An AI Agent Execution Environment to Safeguard User Data

2026-04-21 · Robert Stanley, Avi Verma, Lillian Tsai, Konstantinos Kallas 외 arxiv

AI agents promise to serve as general-purpose personal assistants for their users, which requires them to have access to private user data (e.g., personal and financial information). This poses a serious risk to security…