LQoCo: Learning to Optimize Cache Capacity Overloading in Storage Systems
Cache plays an important role to maintain high and stable performance (i.e. high throughput, low tail latency and throughput jitter) in storage systems. Existing rule-based cache management methods, coupled with engineers' manual configurations, cannot meet ever-growing requirements of both time-varying workloads and complex storage systems, leading to frequent cache overloading. In this paper, we for the first time propose a light-weight learning-based cache bandwidth control technique, called \LQoCo which can adaptively control the cache bandwidth so as to effectively prevent cache overloading in storage systems. Extensive experiments with various workloads on real systems show that LQoCo, with its strong adaptability and fast learning ability, can adapt to various workloads to effectively control cache bandwidth, thereby significantly improving the storage performance (e.g. increasing the throughput by 10\%-20\% and reducing the throughput jitter and tail latency by 2X-6X and 1.5X-4X, respectively, compared with two representative rule-based methods).
Code (0)
등록된 구현이 없습니다.
Tasks
ManagementSimilar Papers 제목 키워드 기반
Long-Term Benefits of Network Boosters for Renewables Integration and Corrective Grid Security
The preventative strategies for $N-1$ network security dominant in European networks mean that network capacity is kept free in case a line fails. If instead fast corrective actions are used to overcome network overloadi…
ObjectCache: Layerwise Object-Storage Retrieval for KV Cache Reuse
Prefix KV caching has become a key mechanism in LLM serving: it reduces time to first token (TTFT) by avoiding redundant computation across requests that share a prefix (i.e., the system prompt). However, the accumulated…
LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference
KV cache has traditionally been stored in GPU memory to accelerate the decoding phase of large language model (LLM) inference. However, it is increasingly necessary to move KV caches outside GPU devices, to enable cache …
Question AnsweringEVICPRESS: Joint KV-Cache Compression and Eviction for Efficient LLM Serving
Reusing KV cache is essential for high efficiency of Large Language Model (LLM) inference systems. With more LLM users, the KV cache footprint can easily exceed GPU memory capacity, so prior work has proposed to either e…
KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference
Language models (LMs) underpin emerging mobile and embedded AI applications like meeting and video summarization and document analysis, which often require processing multiple long-context inputs. Running an LM locally o…
Video Summarization