Agent-X: Full Pipeline Acceleration of On-device AI Agents
LLM-based agents deliver state-of-the-art performance across tasks but incur high end-to-end latency on edge devices. We introduce Agent-X, a software-only, accuracy-preserving framework that accelerates both the prefill and decode stages of on-device agent workloads. Agent-X's two key components rewrite prompts to leverage prefix caching tailored to agent-specific input-token patterns and enable LLM-free speculative decoding for fast token generation with minimal overhead. On representative agentic workloads, Agent-X achieves a 1.61x end-to-end speedup in real systems with no accuracy loss and can be seamlessly integrated into existing on-device AI agents. To the best of our knowledge, ours is the first to systematically characterize and eliminate latency bottlenecks in on-device agents.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents
GUI agents drive applications through their visual interfaces instead of programmatic APIs, interacting with arbitrary software via taps, swipes, and keystrokes, reaching a long tail of applications that CLI-based agents…
AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning
Autonomous agents powered by large language models (LLMs) have garnered significant research attention. However, fully harnessing the potential of LLMs for agent-based tasks presents inherent challenges due to the hetero…
SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
Smartphone agents are increasingly important for helping users control devices efficiently, with (Multimodal) Large Language Model (MLLM)-based approaches emerging as key contenders. Fairly comparing these agents is esse…
AI AgentBenchmarkingLanguage ModellingLarge Language Model+1MobiAgent: A Systematic Framework for Customizable Mobile Agents
With the rapid advancement of Vision-Language Models (VLMs), GUI-based mobile agents have emerged as a key development direction for intelligent mobile systems. However, existing agent models continue to face significant…
LiteGUI: Distilling Compact GUI Agents with Reinforcement Learning
Developing lightweight, on-device vision-language GUI agents is essential for efficient cross-platform automated interaction. However, current on-device agents are constrained by limited model capacity, and further perfo…
Reinforcement LearningKnowledge Distillation