CONCORD: Asynchronous Sparse Aggregation for Device-Cloud RAG under Document Isolation
Retrieval-augmented generation (RAG) has emerged as a pivotal technique for improving language models by incorporating external knowledge at inference time. As device-cloud collaborative inference makes it feasible to deploy small language models on edge devices, a new setting arises in which private documents remain on the device and public knowledge resides in the cloud. Privacy and policy constraints often forbid raw document exchange, creating a document-isolated dual-end RAG setting. However, existing methods rely on frequent remote synchronization and dense evidence transfer, limiting throughput under realistic latency and bandwidth conditions. To address this issue, we propose CONCORD, an asynchronous sparse aggregation framework for dual-end RAG under document isolation. CONCORD treats the cloud as an asynchronously arriving evidence source rather than a continuously synchronized co-generator. Specifically, we introduce waiting debt control to decide whether each decoding step should continue waiting for remote participation based on the observed return of waiting. We also design a certificate-guided minimal supplementation mechanism that requests only the remote evidence needed to determine the current greedy decision. Steps that consult the cloud preserve the same greedy token as dense dual-end aggregation, while the remaining steps commit locally without remote evidence. Experiments on Natural Questions and WikiText-2 show that CONCORD improves end-to-end throughput over baselines by $1.66\times$ and $2.15\times$, respectively, while reducing per-token communication by over two orders of magnitude and maintaining comparable answer quality and perplexity.
Code (0)
등록된 구현이 없습니다.
Tasks
Natural QuestionsResults from the Paper
| Rank | Task | Dataset | Model | Metrics |
|---|---|---|---|---|
| #4 | Retrieval | Natural Questions | CONCORD | Queries per second: 1.66 |
Similar Papers 제목 키워드 기반
Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks
Federated Learning (FL) has gained increasing interest in recent years as a distributed on-device learning paradigm. However, multiple challenges remain to be addressed for deploying FL in real-world Internet-of-Things (…
Federated LearningDevice Scheduling and Update Aggregation Policies for Asynchronous Federated Learning
Federated Learning (FL) is a newly emerged decentralized machine learning (ML) framework that combines on-device local training with server-based model synchronization to train a centralized ML model over distributed nod…
Federated LearningSchedulingBuffered Asynchronous Secure Aggregation for Cross-Device Federated Learning
Asynchronous federated learning (AFL) is an effective method to address the challenge of device heterogeneity in cross-device federated learning. However, AFL is usually incompatible with existing secure aggregation prot…
Federated LearningAEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices
Federated Learning (FL) has achieved significant achievements recently, enabling collaborative model training on distributed data over edge devices. Iterative gradient or model exchanges between devices and the centraliz…
Federated LearningTimely Asynchronous Hierarchical Federated Learning: Age of Convergence
We consider an asynchronous hierarchical federated learning (AHFL) setting with a client-edge-cloud framework. The clients exchange the trained parameters with their corresponding edge servers, which update the locally a…
Federated Learning