Prompt-Aware Scheduling for Efficient Text-to-Image Inferencing System
Traditional ML models utilize controlled approximations during high loads, employing faster, but less accurate models in a process called accuracy scaling. However, this method is less effective for generative text-to-image models due to their sensitivity to input prompts and performance degradation caused by large model loading overheads. This work introduces a novel text-to-image inference system that optimally matches prompts across multiple instances of the same model operating at various approximation levels to deliver high-quality images under high loads and fixed budgets.
Code (0)
등록된 구현이 없습니다.
Tasks
SchedulingSensitivitySimilar Papers 제목 키워드 기반
LLM-Upgraded Graph Reinforcement Learning for Carbon-Aware Job Scheduling in Smart Manufacturing
This paper presents \textsc{Luca}, a \underline{l}arge language model (LLM)-\underline{u}pgraded graph reinforcement learning framework for \underline{c}arbon-\underline{a}ware flexible job shop scheduling. \textsc{Luca}…
Reinforcement LearningGraph Neural NetworkAdaptively Robust LLM Inference Optimization under Prediction Uncertainty
We study the problem of optimizing Large Language Model (LLM) inference scheduling to minimize total latency. LLM inference is an online and multi-task service process and also heavily energy consuming by which a pre-tra…
Implicit In-Context Learning: Evidence from Artificial Language Experiments
Humans acquire language through implicit learning, absorbing complex patterns without explicit awareness. While LLMs demonstrate impressive linguistic capabilities, it remains unclear whether they exhibit human-like patt…
In-Context LearningSemantic Scheduling for LLM Inference
Conventional operating system scheduling algorithms are largely content-ignorant, making decisions based on factors such as latency or fairness without considering the actual intents or semantics of processes. Consequent…
FairnessManagementSchedulingSelect Smarter, Not More: Prompt-Aware Evaluation Scheduling with Submodular Guarantees
Automatic prompt optimization (APO) hinges on the quality of its evaluation signal, yet scoring every prompt candidate on the full training set is prohibitively expensive. Existing methods either fix a single evaluation …