Comet: A Communication-efficient and Performant Approximation for Private Transformer Inference
The prevalent use of Transformer-like models, exemplified by ChatGPT in modern language processing applications, underscores the critical need for enabling private inference essential for many cloud-based services reliant on such models. However, current privacy-preserving frameworks impose significant communication burden, especially for non-linear computation in Transformer model. In this paper, we introduce a novel plug-in method Comet to effectively reduce the communication cost without compromising the inference performance. We second introduce an efficient approximation method to eliminate the heavy communication in finding good initial approximation. We evaluate our Comet on Bert and RoBERTa models with GLUE benchmark datasets, showing up to 3.9$\times$ less communication and 3.5$\times$ speedups while keep competitive model performance compared to the prior art.
Code (0)
등록된 구현이 없습니다.
Tasks
Privacy PreservingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity
With the growing use of large language models (LLMs) hosted on cloud platforms to offer inference services, privacy concerns about the potential leakage of sensitive information are escalating. Secure multi-party computa…
Language ModelingLanguage ModellingLarge Language ModelLLMs Can Understand Encrypted Prompt: Towards Privacy-Computing Friendly Transformers
The community explored to build private inference frameworks for transformer-based large language models (LLMs) in a server-client setting, where the server holds the model parameters and the client inputs its private da…
Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts
Mixture-of-experts (MoE) has been extensively employed to scale large language models to trillion-plus parameters while maintaining a fixed computational cost. The development of large MoE models in the distributed scena…
Computational EfficiencyGPUMixture-of-ExpertsMPCFormer: fast, performant and private Transformer inference with MPC
Enabling private inference is crucial for many cloud inference services that are based on Transformer models. However, existing private inference solutions can increase the inference latency by more than 60x or significa…
Knowledge DistillationCoMet: Metaphor-Driven Covert Communication for Multi-Agent Language Games
Metaphors are a crucial way for humans to express complex or subtle ideas by comparing one concept to another, often from a different domain. However, many large language models (LLMs) struggle to interpret and apply met…