Sub-query Fragmentation for Query Analysis and Data Caching in the Distributed Environment
When data stores and users are distributed geographically, it is essential to organize distributed data cache points at ideal locations to minimize data transfers. To answer this, we are developing an adaptive distributed data caching framework that can identify suitable data chunks to cache and move across a network of community cache locations.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
TokenLake: A Unified Segment-level Prefix Cache Pool for Fine-grained Elastic Long-Context LLM Serving
Prefix caching is crucial to accelerate multi-turn interactions and requests with shared prefixes. At the cluster level, existing prefix caching systems are tightly coupled with request scheduling to optimize cache effic…
Atrapos: Real-time Evaluation of Metapath Query Workloads
Heterogeneous information networks (HINs) represent different types of entities and relationships between them. Exploring, analysing, and extracting knowledge from such networks relies on metapath queries that identify p…
Randomization Boosts KV Caching, Learning Balances Query Load: A Joint Perspective
KV caching is a fundamental technique for accelerating Large Language Model (LLM) inference by reusing key-value (KV) pairs from previous queries, but its effectiveness under limited memory is highly sensitive to the evi…
Topical Result Caching in Web Search Engines
Caching search results is employed in information retrieval systems to expedite query processing and reduce back-end server workload. Motivated by the observation that queries belonging to different topics have different…
Information RetrievalRetrievalContinuous Semantic Caching for Low-Cost LLM Serving
As Large Language Models (LLMs) become increasingly popular, caching responses so that they can be reused by users with semantically similar queries has become a vital strategy for reducing inference costs and latency. E…