paper-with-me

홈 › Papers

AEIA-MN: Evaluating the Robustness of Multimodal LLM-Powered Mobile Agents Against Active Environmental Injection Attacks

2025-02-18 · Yurun Chen, Xueyu Hu, Keting Yin, Juncheng Li, Shengyu Zhang

As researchers continuously optimize AI agents to perform tasks more effectively within operating systems, they often neglect to address the critical need for enabling these agents to identify "impostors" within the system. Through an analysis of the agents' operating environment, we identified a potential threat: attackers can disguise their attack methods as environmental elements, injecting active disturbances into the agents' execution process, thereby disrupting their decision-making. We define this type of attack as Active Environment Injection Attack (AEIA). Based on this, we propose AEIA-MN, an active environment injection attack scheme that exploits interaction vulnerabilities in the mobile operating system to evaluate the robustness of MLLM-based agents against such threats. Experimental results show that even advanced MLLMs are highly vulnerable to this attack, achieving a maximum attack success rate of 93% in the AndroidWorld benchmark.

📄 PDF Abstract BibTeX arXiv:2502.13053

Code (0)

등록된 구현이 없습니다.

Tasks

Decision Making

Similar Papers 제목 키워드 기반

Act2Intention: A Benchmark For Developing Active Mobile Agents Through Inferring User Intention from GUI Actions

2026-08-14 · Xiaokai Yan, Jingtao Ding, Yong Li, Zhiwen Yu arxiv

Mobile GUI Agents powered by multimodal large language models (MLLMs) show promise in human-computer intelligence. However, current research primarily focuses on reactive task execution while lacking a comprehensive unde…

MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control

2024-10-23 · Juyong Lee, Dongyoon Hahm, June Suk Choi, W. Bradley Knox 외

Autonomous agents powered by large language models (LLMs) show promising potential in assistive tasks across various domains, including mobile device control. As these agents interact directly with personal information a…

SnapBench: Benchmarking Snap-and-Ask Multimodal Retrieval for Mobile Interactions

2026-08-30 · Zirong Chen, Fuda Ye, Kuan Zhang, Enjun Du 외 hf

Mobile AI acts as a visual oracle, empowering users to snap a picture of something and ask for information. Snap-and-ask retrieval is now one of the most common entry points for mobile AI, yet photos are often blurry, wh…

Automated Trajectory Evaluation for Mobile Agents via Step-Level Consequence Reasoning and Aggregation

2026-08-21 · Pengshuai Yang, Zijing Gao, Xue Yu, Benhui Zhuang 외 arxiv

Evaluating language-guided mobile agents has recently shifted from rule-based to model-based approaches to achieve scalable and automated assessments. However, existing holistic evaluation paradigms process entire trajec…

PRiSM: An Agentic Multimodal Benchmark for Scientific Reasoning via Python-Grounded Evaluation

2025-12-05 · Shima Imani, Seungwhan Moon, Adel Ahmadyan, Lu Zhang 외 arxiv

Evaluating vision-language models (VLMs) in scientific domains like mathematics and physics poses unique challenges that go far beyond predicting final answers. These domains demand conceptual understanding, symbolic rea…

Program Synthesis