Does Vec2Text Pose a New Corpus Poisoning Threat?
The emergence of Vec2Text -- a method for text embedding inversion -- has raised serious privacy concerns for dense retrieval systems which use text embeddings. This threat comes from the ability for an attacker with access to embeddings to reconstruct the original text. In this paper, we take a new look at Vec2Text and investigate how much of a threat it poses to the different attacks of corpus poisoning, whereby an attacker injects adversarial passages into a retrieval corpus with the intention of misleading dense retrievers. Theoretically, Vec2Text is far more dangerous than previous attack methods because it does not need access to the embedding model's weights and it can efficiently generate many adversarial passages. We show that under certain conditions, corpus poisoning with Vec2Text can pose a serious threat to dense retriever system integrity and user experience by injecting adversarial passaged into top ranked positions. Code and data are made available at https://github.com/ielab/vec2text-corpus-poisoning
Code (1)
Tasks
RetrievalSimilar Papers 제목 키워드 기반
Generating Fake Cyber Threat Intelligence Using Transformer-Based Models
Cyber-defense systems are being developed to automatically ingest Cyber Threat Intelligence (CTI) that contains semi-structured data and/or text to populate knowledge graphs. A potential risk is that fake CTI can be gene…
Data PoisoningKnowledge GraphsLanguage ModellingSentenceRAGuard: A Layered Defense Framework for Retrieval-Augmented Generation Systems Against Data Poisoning
Retrieval-Augmented Generation (RAG) systems ground large language models (LLMs) in external corpora, but this reliance exposes them to corpus poisoning: maliciously injected passages that manipulate retrieved evidence. …
Natural QuestionsWhat Doesn't Kill You Makes You Robust(er): How to Adversarially Train against Data Poisoning
Data poisoning is a threat model in which a malicious actor tampers with training data to manipulate outcomes at inference time. A variety of defenses against this threat model have been proposed, but each suffers from a…
Data PoisoningSequential Data Poisoning in LLM Post-Training
LLM post-training proceeds through multiple stages, e.g., supervised fine-tuning (SFT) followed by reinforcement learning from human feedback (RLHF) or direct preference optimization (DPO), where each stage draws data fr…
Reinforcement LearningVulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks
AI-based code generators have become pivotal in assisting developers in writing software starting from natural language (NL). However, they are trained on large amounts of data, often collected from unsanitized online so…
Code GenerationData Poisoning