Efficient Wireless Federated Learning via Low-Rank Gradient Factorization
This paper presents a novel gradient compression method for federated learning (FL) in wireless systems. The proposed method centers on a low-rank matrix factorization strategy for local gradient compression that is based on one iteration of a distributed Jacobi successive convex approximation (SCA) at each FL round. The low-rank approximation obtained at one round is used as a "warm start" initialization for Jacobi SCA in the next FL round. A new protocol termed over-the-air low-rank compression (Ota-LC) incorporating this gradient compression method with over-the-air computation and error feedback is shown to have lower computation cost and lower communication overhead, while guaranteeing the same inference performance, as compared with existing benchmarks. As an example, when targeting a test accuracy of 70% on the Cifar-10 dataset, Ota-LC reduces total communication costs by at least 33% compared to benchmark schemes.
Code (1)
Tasks
Federated LearningLow-rank compressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Secure Federated Matrix Factorization
To protect user privacy and meet law regulations, federated (machine) learning is obtaining vast interests in recent years. The key principle of federated learning is training a machine learning model without needing to …
BIG-bench Machine LearningFederated LearningCommunication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation
In federated learning, fine-tuning pre-trained foundation models poses significant challenges, particularly regarding high communication cost and suboptimal model performance due to data heterogeneity between the clients…
Federated LearningProjected Gradient Descent for Spectral Compressed Sensing via Symmetric Hankel Factorization
Current spectral compressed sensing methods via Hankel matrix completion employ symmetric factorization to demonstrate the low-rank property of the Hankel matrix. However, previous non-convex gradient methods only utiliz…
compressed sensingMatrix CompletionBayesian Federated Learning over Wireless Networks
Federated learning is a privacy-preserving and distributed training method using heterogeneous data sets stored at local devices. Federated learning over wireless networks requires aggregating locally computed gradients …
Federated LearningPrivacy PreservingDeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models
Low-Rank Adaptation (LoRA) has emerged as one of the most effective, computationally tractable fine-tuning approaches for training Vision-Language Models (VLMs) and Large Language Models (LLMs). LoRA accomplishes this by…
Federated Learning