paper-with-me

홈 › Papers

Forensic Trajectory Signatures for Agent Memory Poisoning Detection

2026-06-29 · Jun Wen Leong arxiv

We discover a behavioral invariant in LLM agents under persistent memory poisoning: in architectures where routing information is retrieved through observable memory-tool invocations, successful attacks require calling memory_recall_fact before email_send_email, a transition that non-exfiltrating sessions rarely exhibit. Under the evaluated architecture, this invariant follows from the attack's information-retrieval dependency rather than being merely an empirical correlation, and suppressing it breaks the attack. A simple rule exploiting this invariant alone achieves AUC = 0.9563. A Random Forest classifier over 19 trajectory features refines it to AUC = 0.9904 (BCa 95% CI [0.987, 0.993], N=10,000 resamples), demonstrating that the attack imprints on multiple independent behavioral channels. The signature is overdetermined: removing all recall-related features (half the feature set) leaves AUC unchanged at 0.990, confirming that memory poisoning induces a distributed trajectory signature rather than a single observable anomaly. Cross-model hold-out on 9 models (7B-120B parameters) confirms AUC = 1.000 on 6/9 hold-out splits, with all three exceptions mechanistically explained. The invariant generalizes to frontier models (GPT-4.1, GPT-4o) without retraining. A strictly prefix-only variant achieves AUC = 0.934, suggesting that real-time blocking is feasible with moderate degradation. The boundary is forensically useful: prompt-injection attacks that bypass memory produce a distinct trajectory (score = 0.541), enabling incident responders to distinguish memory-channel attacks from prompt-injection attacks using tool-call logs alone.

📄 PDF Abstract BibTeX arXiv:2606.30566

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agents

2026-04-03 · Wei Zou, Mingwen Dong, Miguel Romero Calvo, Shuaichen Chang 외 arxiv

Memory makes LLM-based web agents personalized, powerful, yet exploitable. By storing past interactions to personalize future tasks, agents inadvertently create a persistent attack surface that spans websites and session…

From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents

2026-06-03 · Pritam Dash, Tongyu Ge, Aditi Jain, Tanmay Shah 외 arxiv

Memory is a core component of AI agents, enabling them to accumulate knowledge across interactions and improve performance. However, persistent memory introduces the risk of memory poisoning, where a single adversarial m…

Tracing Back the Malicious Clients in Poisoning Attacks to Federated Learning

2024-07-09 · Yuqi Jia, Minghong Fang, Hongbin Liu, Jinghuai Zhang 외

Poisoning attacks compromise the training phase of federated learning (FL) such that the learned global model misclassifies attacker-chosen inputs called target inputs. Existing defenses mainly focus on protecting the tr…

Federated Learning

The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems

2026-05-12 · Tanzim Ahad, Ismail Hossain, Md Jahangir Alam, Sai Puppala 외 arxiv

Multi-agent AI pipelines typically assume that agent misconduct originates from model misalignment. We identify a structural failure in this assumption, the \emph{Misattribution Gap}, where memory-layer attacks produce b…

Memory poisoning and secure multi-agent systems

2026-03-20 · Vicenç Torra, Maria Bras-Amorós arxiv

Memory poisoning attacks for Agentic AI and multi-agent systems (MAS) have recently caught attention. It is partially due to the fact that Large Language Models (LLMs) facilitate the construction and deployment of agents…