paper-with-me

Papers

Your Agent is More Brittle Than You Think: Uncovering Indirect Injection Vulnerabilities in Agentic LLMs

2026-04-04 · Wenhui Zhu, Xuanzhao Dong, Xiwen Chen, Rui Cai, Peijie Qiu, Zhipeng Wang, Oana Frunza, Shao Tang, Jindong Gu, Yalin Wang arxiv

The rapid deployment of open-source frameworks has significantly advanced the development of modern multi-agent systems. However, expanded action spaces, including uncontrolled privilege exposure and hidden inter-system interactions, pose severe security challenges. Specifically, Indirect Prompt Injections (IPI), which conceal malicious instructions within third-party content, can trigger unauthorized actions such as data exfiltration during normal operations. While current security evaluations predominantly rely on isolated single-turn benchmarks, the systemic vulnerabilities of these agents within complex dynamic environments remain critically underexplored. To bridge this gap, we systematically evaluate six defense strategies against four sophisticated IPI attack vectors across nine LLM backbones. Crucially, we conduct our evaluation entirely within dynamic multi-step tool-calling environments to capture the true attack surface of modern autonomous agents. Moving beyond binary success rates, our multidimensional analysis reveals a pronounced fragility. Advanced injections successfully bypass nearly all baseline defenses, and some surface-level mitigations even produce counterproductive side effects. Furthermore, while agents execute malicious instructions almost instantaneously, their internal states exhibit abnormally high decision entropy. Motivated by this latent hesitation, we investigate Representation Engineering (RepE) as a robust detection strategy. By extracting hidden states at the tool-input position, we revealed that the RepE-based circuit breaker successfully identifies and intercepts unauthorized actions before the agent commits to them, achieving high detection accuracy across diverse LLM backbones. This study exposes the limitations of current IPI defenses and provides a highly practical paradigm for building resilient multi-agent architectures.

📄 PDF Abstract BibTeX arXiv:2604.03870

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Perspectra: Choosing Your Experts Enhances Critical Thinking in Multi-Agent Research Ideation

2025-09-24 · Yiren Liu, Viraj Shah, Sangho Suh, Pao Siangliulue 외 arxiv

Recent advances in multi-agent systems (MAS) enable tools for information search and ideation by assigning personas to agents. However, how users can effectively control, steer, and critically evaluate collaboration amon…

ReThinker: Scientific Reasoning by Rethinking with Guided Reflection and Confidence Control

2026-02-04 · Zhentao Tang, Yuqi Cui, Shixiong Kai, Wenqian Zhao 외 arxiv

Expert-level scientific reasoning remains challenging for large language models, particularly on benchmarks such as Humanity's Last Exam (HLE), where rigid tool pipelines, brittle multi-agent coordination, and inefficien…

HugAgent: Benchmarking LLMs for Simulation of Individualized Human Reasoning

2025-10-16 · Chance Jiajie Li, Zhenze Mo, Yuhan Tang, Ao Qu 외 arxiv

Simulating human reasoning in open-ended tasks has long been a central aspiration in AI and cognitive science. While large language models now approximate human responses at scale, they remain tuned to population-level c…

How Brittle is Agent Safety? Rethinking Agent Risk under Intent Concealment and Task Complexity

2025-11-11 · Zihan Ma, Dongsheng Zhu, Shudong Liu, Taolin Zhang 외 arxiv

Current safety evaluations for LLM-driven agents primarily focus on atomic harms, failing to address sophisticated threats where malicious intent is concealed or diluted within complex tasks. We address this gap with a t…

How to Train Your Deep Research Agent? Prompt, Reward, and Policy Optimization in Search-R1

2026-02-23 · Yinuo Xu, Shuo Lu, Jianjie Cheng, Meng Wang 외 arxiv

Deep Research agents tackle knowledge-intensive tasks through multi-round retrieval and decision-oriented generation. While reinforcement learning (RL) has been shown to improve performance in this paradigm, its contribu…

Reinforcement Learning