CtrlRAG: Black-box Adversarial Attacks Based on Masked Language Models in Retrieval-Augmented Language Generation
Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by integrating external knowledge bases. However, this integration introduces a new security threat: adversaries can exploit the retrieval mechanism to inject malicious content into the knowledge base, thereby influencing the generated responses. Based on this attack vector, we propose CtrlRAG, a novel attack method designed for RAG system in the black-box setting, which aligns with real-world scenarios. Unlike existing attack methods, CtrlRAG introduces a perturbation mechanism using Masked Language Model (MLM) to dynamically optimize malicious content in response to changes in the retrieved context. Experimental results demonstrate that CtrlRAG outperforms three baseline methods in both Emotional Manipulation and Hallucination Amplification objectives. Furthermore, we evaluate three existing defense mechanisms, revealing their limited effectiveness against CtrlRAG and underscoring the urgent need for more robust defenses.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationLanguage ModelingLanguage ModellingRAGRetrievalRetrieval-augmented GenerationText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Context Aware Approach for Generating Natural Language Attacks
We study an important task of attacking natural language processing models in a black box setting. We propose an attack strategy that crafts semantically similar adversarial examples on text classification and entailment…
Language ModellingSentencetext-classificationText ClassificationImproving Robustness of Facial Landmark Detection by Defending Against Adversarial Attacks
Many recent developments in facial landmark detection have been driven by stacking model parameters or augmenting annotations. However, three subsequent challenges remain, including 1) an increase in computational ov…
Face AlignmentFacial Landmark DetectionText Adversarial Purification as Defense against Adversarial Attacks
Adversarial purification is a successful defense mechanism against adversarial attacks without requiring knowledge of the form of the incoming attack. Generally, adversarial purification aims to remove the adversarial pe…
Adversarial AttackAdversarial DefenseAdversarial PurificationArabic Synonym BERT-based Adversarial Examples for Text Classification
Text classification systems have been proven vulnerable to adversarial text examples, modified versions of the original text examples that are often unnoticed by human eyes, yet can force text classification models to al…
Adversarial TextLanguage ModelingLanguage ModellingMasked Language Modeling+2Masked Language Model Based Textual Adversarial Example Detection
Adversarial attacks are a serious threat to the reliable deployment of machine learning models in safety-critical applications. They can misguide current models to predict incorrectly by slightly modifying the inputs. Re…
Adversarial DefenseLanguage ModelingLanguage ModellingSST-2