Towards Scalable Customization and Deployment of Multi-Agent Systems for Enterprise Applications
Large language model (LLM)-based multi-agent systems demonstrate strong performance on complex reasoning and task execution, enabling broad enterprise applications. However, production deployment remains challenging due to domain-specific customization requirements and high latency and inference costs in agentic workflows. We propose a unified framework for customization and efficient deployment of multi-agent systems in real-world settings. The first stage, Agentic Model Customization, combines continual pretraining, supervised fine-tuning, and preference optimization to adapt a compact model to specialized domains while retaining strong agentic capabilities. The second stage, Inference Optimization, integrates speculative decoding and FP8 quantization with targeted calibration to enable cost-efficient serving with minimal quality loss. Across enterprise workloads, our framework enables rapid domain adaptation and achieves a 4.48x speedup in throughput while maintaining performance and improving robustness on long-tail scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual PretrainingDomain AdaptationSimilar Papers 제목 키워드 기반
AutoMCU: Feasibility-First MCU Neural Network Customization via LLM-based Multi-Agent Systems
Deploying neural networks on microcontroller units (MCUs) is critical for edge intelligence but remains challenging due to tight memory, storage, and computation constraints. Existing approaches, such as model compressio…
Neural Architecture SearchModel CompressionPosition: Vector Prompt Interfaces Should Be Exposed to Enable Customization of Large Language Models
As large language models (LLMs) transition from research prototypes to real-world systems, customization has emerged as a central bottleneck. While text prompts can already customize LLM behavior, we argue that text-only…
Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration
Microprocessor architects are increasingly resorting to domain-specific customization in the quest for high-performance and energy-efficiency. As the systems grow in complexity, fine-tuning architectural parameters acros…
Compiler OptimizationMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+1Trade in Minutes! Rationality-Driven Agentic System for Quantitative Financial Trading
Recent advancements in large language models (LLMs) and agentic systems have shown exceptional decision-making capabilities, revealing significant potential for autonomic finance. Current financial trading agents predomi…
Mathematical ReasoningTailoring with Targeted Precision: Edit-Based Agents for Open-Domain Procedure Customization
How-to procedures, such as how to plant a garden, are now used by millions of users, but sometimes need customizing to meet a user's specific needs, e.g., planting a garden without pesticides. Our goal is to measure and …