Hybrid-RACA: Hybrid Retrieval-Augmented Composition Assistance for Real-time Text Prediction
Large language models (LLMs) enhanced with retrieval augmentation has shown great performance in many applications. However, the computational demands for these models pose a challenge when applying them to real-time tasks, such as composition assistance. To address this, we propose Hybrid Retrieval-Augmented Composition Assistance (Hybrid-RACA), a novel system for real-time text prediction that efficiently combines a cloud-based LLM with a smaller client-side model through retrieval augmented memory. This integration enables the client model to generate better responses, benefiting from the LLM's capabilities and cloud-based data. Meanwhile, via a novel asynchronous memory update mechanism, the client model can deliver real-time completions to user inputs without the need to wait for responses from the cloud. Our experiments on five datasets demonstrate that Hybrid-RACA offers strong performance while maintaining low latency.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationLanguage ModellingLarge Language ModelRetrievalSimilar Papers 제목 키워드 기반
RACap: Relation-Aware Prompting for Lightweight Retrieval-Augmented Image Captioning
Recent retrieval-augmented image captioning methods incorporate external knowledge to compensate for the limitations in comprehending complex scenes. However, current approaches face challenges in relation modeling: (1) …
Image CaptioningRACAI GEC -- A hybrid approach to Grammatical Error Correction
Agentic Hybrid RAG for Evidence-Grounded Muon Collider Analysis
Muon collider research spans accelerator physics, detector instrumentation, and high-energy phenomenology, with relevant evidence scattered across a rapidly expanding and heterogeneous body of scientific literature. As h…
Semantic RetrievalQuestion AnsweringAnswer GenerationDAT: Dynamic Alpha Tuning for Hybrid Retrieval in Retrieval-Augmented Generation
Hybrid retrieval techniques in Retrieval-Augmented Generation (RAG) systems enhance information retrieval by combining dense and sparse (e.g., BM25-based) retrieval methods. However, existing approaches struggle with ada…
Information RetrievalLanguage ModelingLanguage ModellingLarge Language Model+3Compositional Design of Safety Controllers for Large-scale Stochastic Hybrid Systems
In this work, we propose a compositional scheme based on small-gain reasoning for the safety controller synthesis of interconnected stochastic hybrid systems with both continuous evolutions and instantaneous jumps. In ou…